**Location**: Remote
**Type**: Full-time or Part-time Contract Work
**Fluent Language Skills Required**: English & Brazilian Portuguese
**Why This Role Exists**:
This organization works with AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems.
These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions.
This project focuses on evaluating and **improving general chat behavior** in large language models (LLMs).
You will assess model-generated responses across diverse topics, provide high-quality human feedback, and help ensure AI systems communicate in ways that are accurate, well-reasoned, and aligned with human expectations.
**What You'll Do**:
- Evaluate LLM-generated responses on their ability to effectively answer user queries
- Conduct fact-checking using trusted public sources and external tools
- Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies
- Assess reasoning quality, clarity, tone, and completeness of responses
- Ensure model responses align with expected conversational behavior and system guidelines
**Who You Are**:
- You hold a **Bachelor's degree**




- You are a **native speaker** or have **ILR 5/primary fluency (C2 on the CEFR scale)** in **Brazilian Portuguese**
- You have **significant experience using large language models** (LLMs) and understand how and why people use them
- You have **excellent writing skills** and can clearly articulate nuanced feedback
- You have **strong attention to detail** and consistently notice subtle issues others may overlook
- You are **adaptable** and comfortable moving across topics, domains, and customer requirements
- You have a background or experience in domains requiring **structured analytical thinking** (e.g., research, policy, analytics, linguistics, engineering)
- You have **excellent college-level mathematics skills**
**Nice-to-Have Specialties**:
- Prior experience with **RLHF, model evaluation, or data annotation work**
- Experience writing or editing **high-quality written content**
- Experience comparing multiple outputs and making **fine-grained qualitative judgments**
- **Familiarity with evaluation rubrics**, benchmarks, or quality scoring systems
**What Success Looks Like**:
- You identify factual inaccuracies, reasoning errors, and communication gaps in model responses
- You produce clear, consistent, and reproducible evaluation artifacts
- Your feedback leads to measurable improvements in response quality and user experience

📌 Brazilian Portuguese Expert - Remote (México)
🏢 YO IT CONSULTING
📍 México