Job Type: Flexible Contract (Remote), Mexico
Job Summary: We are hiring PhD-level LLM Post-Training and Evaluation Researchers for a contract project that tests whether frontier AI models can handle real post-training and evaluation work. You will create hard tasks from your field that a leading AI model cannot solve, then review where and why it fails.

What you will do

- Create command line tasks based on real LLM post-training and evaluation problems. Examples:
- Implement a post-training method from a recent paper, such as SFT, DPO, PPO or GRPO, and reproduce its reported results.
- Find the bug in a fine-tuning or RLHF pipeline, such as a wrong loss mask, a broken chat template, a reward model leak or an incorrect KL penalty.
- Debug a failing training run, such as reward hacking, mode collapse or diverging loss.
- Find contamination, leakage or a scoring error in an evaluation benchmark, and fix the harness.
- Build or repair an automated grader, such as an LLM-as-judge or a rule-based verifier, and show where it can be gamed.
- Optimize a slow data pipeline, rollout or inference job without changing its results.
- Build each task’s environment, including the code, data and dependencies, and write a reference solution.
- Write automated tests that check whether a solution is correct, with consistent results on every run (fixed seeds, tolerance-based metric checks).
- Run your task against a leading AI model, review its attempt, and adjust the task until the model fails because the problem is truly hard, not because the instructions are unclear.

Who we are looking for

- A PhD in machine learning, computer science, statistics, applied mathematics, engineering or another STEM field.



PhD candidates and students are welcome to apply.
- Hands-on research or industry experience in LLM post-training, fine-tuning, alignment or evaluation. Ten or more years of experience in machine learning or a related field is adecuado.
- At least one publication, academic or professional.
- Strong experience with PyTorch or JAX, including debugging real training runs.
- Familiarity with tools such as Hugging Face Transformers, TRL, vLLM, DeepSpeed or similar is a plus.
- Comfort with Python, the Linux command line, Docker and Git.
- Experience with experimental design, benchmarking or testing AI systems is a plus.
- Experience with reinforcement learning, reward modeling or distributed training is a plus.
- Peer review experience, or experience building technical tests or assessments, is an advantage.
- Able to work independently in a remote setting and explain technical findings clearly.
- Based in Mexico.

Why Cobalt AI

- Advance frontier AI where it counts. Apply your expertise to the data frontier labs cannot get any other way, where your reasoning directly shapes how the next generation of models solves technical problems.
- Grow professionally. Expand your influence through evaluation projects, advisory roles and research collaborations, while deepening your understanding of how frontier models are trained and assessed.
- Work with a top-tier network. Collaborate with researchers and engineers from leading institutions and labs on high-impact, flexible work.
- Set your own schedule. Flexible 10 to 40 hour weeks that fit around your current role and your life.
- Competitive pay. Paid per accepted task, with higher rates for consistently strong work.

Job Type: Part-time

Work Location: Remote

📌 LLM Post-Training and evaluation researcher (Mexico) (México)
🏢 COBALT
📍 México