**This opportunity is available across all Player Engagement locations and is intended for internal hiring only.
**
**Role Overview**
As Keywords continues to expand Helpshift Care AI across Player Engagement, quality will become one of the most important factors in successful AI adoption.
The AI Ops Quality Manager plays a critical role in ensuring Care AI performs accurately, safely, and in alignment with each client's tone, standards, and player experience expectations.
This role supports AI Ops Managers and Platform Leads by owning the quality discipline around live AI performance.
As more clients move into launch, hypercare, BAU, and future AI feature adoption, this role ensures AI agent performance is not only monitored, but continuously improved through structured review, feedback, root cause analysis, and measurable action.
The AI Ops Quality Manager sits at the intersection of Operations, Product, Development, Account Management, Marketing, and Client Success.
This is a role for someone who can protect quality while helping the business move faster.
**What are we looking for?
**
**Quality & Governance**
- Design, maintain, and continuously improve the QA framework for Helpshift Care AI across launch, hypercare, live client environments, and future AI product capabilities.
- Create clear quality standards for AI agent responses, including accuracy, completeness, tone alignment, escalation behavior, safety, and player experience.
- Define scoring rubrics, audit methods, review cadences, and reporting formats that allow AI performance to be measured consistently across clients.
- Develop QA criteria, test scenarios, audit checks, and feedback loops for new AI features to ensure they are accurate, stable, client-ready, and aligned with player experience expectations.




- Help ensure that as the Care AI roadmap expands, Keywords has a structured quality discipline in place to support new features without creating unnecessary risk for clients, players, or live operations.
**Analytics**
- Conduct regular audits of AI agent interactions on live accounts to identify performance trends, defects, risks, and improvement opportunities.
- Assess whether AI responses are accurate, helpful, aligned to the client's voice, and appropriate for the player's issue type.
- Identify recurring gaps such as incorrect answers, weak source content, poor escalation logic, low-confidence responses, hallucination risk, or tone mismatch.
- Use a structured DMAIC approach to improve AI agent performance on live accounts.
- Define the quality issue, measure the current performance, analyze the root cause, implement improvements, and control the process through ongoing monitoring.
- Interpret quality data, identify trends, and recommend practical actions that improve AI performance and player experience.
**Communication**
- Act as a key guardian of client tone and player experience within Care AI deployments.
- Evaluate whether AI agent responses reflect each client's brand, communication style, support philosophy, and player expectations.
- Partner with AI Ops Managers, Platform Leads, and client teams to ensure AI performance supports trust rather than simply increasing automation volume.




- Serve as a proactive feedback partner to Product and Development teams by translating live account quality findings into actionable insights.
- Document recurring AI performance gaps, platform limitations, client-impacting issues, and enhancement opportunities.
- Provide quality insights, performance summaries, risk flags, and improvement updates to AI Ops leadership and key stakeholders.
- Help translate detailed QA findings into clear business language for Account Management, Client Success, Marketing, Product, and executive audiences.
**Business**
- Partner with Product, Development, Platform Leads, and Operations to convert quality findings into practical fixes, content improvements, workflow changes, and product feedback.
- Help prioritize improvements based on client impact, player experience risk, commercial value, and operational urgency.
- Support the QA process for new Helpshift Care AI capabilities and upcoming AI product feature sets as they move through the product roadmap.
- Partner with Product, Development, AI Ops Managers, and Platform Leads to understand intended feature behavior, client use cases, operational impact, and quality risks before new capabilities are released into live environments.
- Validate new feature sets against real support workflows, client tone requirements, escalation paths, and operational use cases so AI Ops can confidently support adoption after release.
- Document findings, identify defects or performance gaps, and provide clear recommendations to Product and Development teams before broader rollout.
- Highlight wins, client-ready proof points, and measurable improvements that can support adoption, expansion, and commercial storytelling.
**Leadership**
- Partner with AI Ops Managers and Platf

📌 Internal Recruitment) - Ai Ops Quality Manager, Helpshift Care Ai (Xico)
🏢 Keywords Studios
📍 Xico