Job Title: GenAI/ML Evaluation Engineer Experience: 3–8 Years Openings: 1–2 Focus: GenAI/ML Evaluation, Data Pipelines, Test Infrastructure, NLP Location: Mexico, Remote Role Overview We are looking for an engineer to build an automated evaluation and regression framework for a PII rewriting/sanitization service . The role involves building Golden Datasets, processing human annotations, developing evaluation pipelines, and measuring PII protection and rewrite quality. Key Responsibilities Build and maintain Golden Datasets from human annotations. Develop Python/data pipelines to extract and process data from Google Docs, Sheets, Slides, Gmail, Calendar, Chat, PDF, and Office files. Design data schemas/protobufs for machine-readable evaluation data. Build automated evaluation and regression test frameworks . Implement precision, recall, entity-level accuracy, mention-level accuracy, and rewrite consistency metrics. Develop automated reporting and regression detection. Support multilingual evaluation across English, Spanish, Portuguese, Japanese, and Korean. Use GenAI/LLMs for evaluation, annotation validation,



error analysis, and quality measurement. Integrate evaluation frameworks with existing test/CI infrastructure. Must-Have Skills 3 years of software engineering experience with Python, C++, Java, or Go. Strong Python and data-processing experience. Experience building test automation, regression frameworks, or data pipelines . Hands-on experience with NLP, ML, GenAI, or LLM-based systems . Strong understanding of precision, recall, false positives/negatives, and evaluation metrics . Experience with APIs, structured data, protobufs, and large datasets . Experience building or working with evaluation datasets/benchmarks . Strong debugging, data analysis, and problem-solving skills. Preferred GenAI/LLM evaluation or benchmarking experience. PII detection, anonymization, or data privacy experience. Document parsing or Google Workspace API experience. Multilingual NLP experience. CI/CD and large-scale test infrastructure experience.

📌 GenAI/ML Evaluation Engineer (México)
🏢 Centraprise
📍 México