Description AI Operations Specialist – LLM & Conversation Systems
Full-Time, Remote Working Hours:
Business Hours
AI Operations Specialist
to own and improve our production AI conversation systems. We are an AI-powered SMS lead recovery platform serving telehealth brands, processing millions of messages each month. Our AI systems interact directly with customers, and their performance directly impacts client conversions. run LLM systems in production
and understands how to continuously monitor, troubleshoot, test, and improve them.
Ensure new client systems are properly tested before going live. Troubleshoot integration and configuration issues during onboarding. Production AI Operations
Identify issues affecting conversation quality and conversion. Adjust system instructions and conversation logic based on real-world performance. Test changes systematically before and after deployment. Performance & Conversion Analysis
Analyze conversation transcripts and performance data. Track key performance outcomes and communicate findings to leadership.



Use data and real customer interactions to guide optimization decisions. LLM or AI systems in production . Hands-on experience monitoring and improving production AI outputs. Strong understanding of LLM behavior, failure modes, and iterative testing. Excellent written English — required for editing customer-facing AI messages and documenting system changes. Ability to work independently and take ownership of production systems.
You've actually operated AI systems in production rather than only experimenting with ChatGPT. You are methodical about testing changes and measuring their impact. You'll begin by reviewing the performance of live AI conversations and looking for unusual patterns, failures, or opportunities for improvement. You'll also review conversation transcripts, identify issues with AI responses, update prompts or system instructions, test your changes, and monitor the resulting conversion performance. Throughout

📌 Manager, Operations/Production (Monterrey)
🏢 Pavago
📍 Monterrey