iMerit is seeking AI Response Evaluation Analyst to participate in a project where you will review AI generated responses and decide which ones actually hold up: accurate, relevant, concise, safe, well localized, and true to what the user asked for. Every ranking comes with a short written rationale, and those comments feed directly into how the models improve.
Careful reading and good judgment matter far more than speed here.
What you will do
Interpret conversational context and identify what the user really wanted
Rate and rank responses against defined quality criteria
Compare multiple answers and explain in writing why one wins
Verify factual claims using approved research sources
Flag tasks that cannot be reliably assessed rather than guessing
What you bring
Strong critical thinking and sound judgment in ambiguous cases
Solid research skills and attention to detail
Excellent reading comprehension
Professional written English that is clear and concise
Self direction and the discipline to hit deadlines without supervision
Prior experience in AI evaluation, annotation, writing, editing, research, or translation is a plus, though not required.
Good to know
Independent contractor engagement for the length of the project.
Fully remote and adaptable. You choose your hours as long as volume and deadlines are met.
Task volume varies with project demand.
A paid qualification assessment is required before live work begins.