Hello,

Greetings from ZettaMine

Freelancing Opportunity :- We are currently hiring AI Benchmark Quality Reviewers for a short-term AI evaluation and benchmark quality project across LATAM.

Role: AI Benchmark Quality Reviewer

Experience: 5+ Years of Relevant Experience

Location: LATAM

Mode of Work: Remote

Engagement: Full-Time – 40 Hours/Week

Start Date: Immediate

PST Overlap: 8 Hours/Day

Description

We are seeking experienced AI Benchmark Quality Reviewers to validate the quality, correctness, and fairness of AI benchmark tasks and evaluation systems.
The role involves reviewing task instructions, reference solutions, grading criteria, agent execution traces, tool calls, and generated deliverables to identify task defects, grading issues, environment failures, and other quality concerns.

Responsibilities
• Validate task quality by checking instructions, source materials, reference solutions, and evaluation criteria for consistency.
• Identify hidden requirements, missing information, ambiguous instructions, or other task defects.




• Review agent performance by inspecting execution traces, tool calls, and generated deliverables.
• Determine whether agent successes and failures are appropriately justified.
• Audit grading logic and identify brittle checks, incorrect expected answers, unsupported rubric criteria, or unfair evaluation conditions.
• Identify cases where valid alternative solutions may be incorrectly penalized.
• Investigate discrepancies between model performance, task defects, grader errors, and environment or tool failures.
• Independently assess automated QC findings rather than accepting them at face value.
• Document concise, evidence-backed findings and actionable feedback.
• Flag uncertainty and verify that revisions successfully resolve identified issues.

Requirements
• 5+ years of relevant professional experience.
• Comfortable reading Python, SQL, shell scripts, structured data, and execution logs to understand task setup and gradin

📌 AI Benchmark Quality Reviewer (México)
🏢 ZettaMine Labs
📍 México