We are looking for experienced Subject Matter Experts (SMEs) to support an AI conversation evaluation project.
The role involves reviewing AI conversations involving real-world interpersonal and everyday decision-making scenarios, identifying missing context, and providing structured expert feedback to improve conversational quality.
What You’ll Do
- Review AI-generated conversations and recommendations
- Identify important context that may have been missed
- Suggest relevant follow-up questions that could improve the quality of recommendations
- Assess whether additional information could affect an AI recommendation
- Distinguish important information from non-essential details
- Conduct structured qualitative reviews and document findings
- Collaborate with research teams to maintain consistent evaluation standards 108815 - Text Situated Alignm…
Who We're Looking For Professionals with 3-5 years of direct relevant experience in one or more of the following areas:
- Social Work - MSW and relevant professional license/credential
- Counseling / Psychology - relevant graduate qualification, PhD/PsyD, or applicable professional credential
- Qualitative Research / HCI / Communication
- Anthropology / Sociology
- Mediation / Coaching
- Student Services
- Organizational Consulting or related advisory fields 108815 - Text Situated Alignm…
Preferred Experience
- Client or participant interviewing
- Qualitative research or case review
- Conversation analysis or qualitative coding
- Experience gathering information when initial context is incomplete
- Ability to identify information that is critical to a recommendation
- Experience reviewing consultations, interviews, coaching sessions, or similar interactions