imerit is seeking ai response evaluation analyst to participate in a project where you will review ai generated responses and decide which ones actually hold up: accurate, relevant, concise, safe, well localized, and true to what the user asked for. Every ranking comes with a short written rationale, and those comments feed directly into how the models improve.
careful reading and good judgment matter far more than speed here.
what you will do
- interpret conversational context and identify what the user really wanted
- rate and rank responses against defined quality criteria
- compare multiple answers and explain in writing why one wins
- verify factual claims using approved research sources
- flag tasks that cannot be reliably assessed rather than guessing
what you bring
- strong critical thinking and sound judgment in ambiguous cases
- solid research skills and attention to detail
- excellent reading comprehension
- professional written english that is clear and concise
- self direction and the discipline to hit deadlines without supervision
prior experience in ai evaluation, annotation, writing, editing, research, or translation is a plus, though not required.
good to know
- independent contractor engagement for the length of the project.
- fully remote and versátil. You choose your hours as long as volume and deadlines are met.
- task volume varies with project demand.
- a paid qualification assessment is required before live work begins.