18 nov
|
Programming.Com
|
México
18 nov
Programming.Com
México
Position: AI CSV / CSA Validation Analyst
Employment Type: Contract
Location:remote
Project: GxP AI / GenAI Applications Portfolio
Job Overview
We are seeking an experienced AI CSV / CSA Validation Analyst to support the validation of custom GxP AI and Generative AI applications within a regulated life sciences environment.
The adecuado candidate will have strong hands-on experience in Computer System Validation (CSV), Computer Software Assurance (CSA), and GxP compliance, combined with a practical understanding of LLMs, Retrieval-Augmented Generation (RAG), AI agents, and probabilistic AI system testing.
This role focuses on translating AI technical risks and failure modes into defensible validation requirements, risk controls, test strategies, supporting evidence, and release decisions. The consultant will collaborate closely with AI Engineering, Business stakeholders, CSV, and Quality Assurance teams.
Key Responsibilities
- Develop and coordinate risk-based assessments and validation strategies for custom GxP AI/GenAI applications.
- Assess GxP intended use, electronic records, regulatory impact, patient safety, and data integrity risks.
- Prepare and coordinate validation documentation, including validation plans, user requirements, functional requirements, traceability matrices, IQ/OQ/PQ or equivalent testing protocols, deviations, and validation reports.
- Translate AI-specific technical risks into testable requirements, controls, and acceptance criteria.
- Design and execute risk-based evaluation strategies for probabilistic AI systems using representative test datasets, variability checks, negative testing, and measurable pass/fail criteria.
- Evaluate AI outputs for factual accuracy, completeness, relevance, consistency, citation accuracy, source grounding, and retrieval quality.
- Assess AI-specific risks, including hallucinations, unsupported claims, omissions, prompt injection, inappropriate agent actions, and excessive autonomy.
- Evaluate human review, approval controls,
and intended-use boundaries for AI-generated outputs.
- Assess data provenance, source quality, access controls, transformations, versioning, audit trails, and end-to-end traceability.
- Review system, interface, and end-to-end testing, including downstream agent behavior following model, prompt, retrieval, or source-data changes.
- Define regression testing, change-control requirements, approvals, and ongoing monitoring based on risk.
- Coordinate with Business, AI Engineering, CSV, and QA teams to resolve validation gaps and ensure audit-ready documentation.
- Track validation deliverables, identify missing evidence, escalate material risks, and support release-readiness decisions.
Required Qualifications 1. GxP CSV / CSA Validation
- Demonstrated hands-on experience delivering Computer System Validation (CSV) and Computer Software Assurance (CSA) activities.
- Strong experience with regulatory risk assessments, GxP intended-use assessments, and electronic-record assessments.
- Experience preparing validation plans, requirements, traceability matrices, IQ/OQ/PQ or equivalent protocols, deviation reports, evidence reviews, validation summaries, and change controls.
- Familiarity with GAMP 5, FDA CSA guidance, and 21 CFR Part 11 principles.
2. Generative AI / LLM Architecture
- Practical understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI agent architectures.
- Understanding of document ingestion, chunking, embeddings, retrieval, grounding, and downstream integrations.
- Familiarity with prompts, tool calls, agent orchestration, and AI application workflows.
3.
Probabilistic AI System Testing
- Ability to distinguish deterministic requirements from nondeterministic AI-generated outputs.
- Experience developing repeatable, risk-based AI testing and evaluation strategies.
- Knowledge of representative test datasets, variability testing, negative testing, and defensible acceptance criteria.
4. AI Risk Assessment and Output Evaluation
- Ability to assess hallucinations, unsupported claims, omissions, prompt injection, and inappropriate agent actions.
- Understanding of human-in-the-loop controls, review and approval workflows, and AI autonomy risks.
- Experience designing AI output evaluations for factual correctness, completeness, relevance, consistency, citation accuracy, and retrieval quality.
- Ability to establish measurable acceptance thresholds for clinical and regulatory use cases.
5. Data Integrity and Traceability
- Experience assessing data quality, source provenance, access controls, transformations, and version management.
- Understanding of audit trails and traceability from source documents through retrieved context to generated AI outputs.
- Knowledge of GxP data integrity principles.
6. Lifecycle and Change Control
- Experience assessing validation impacts resulting from model, prompt, retrieval-index, source-data, configuration, and integration changes.
- Ability to define risk-based regression testing, approvals, and ongoing monitoring requirements.
Preferred Qualifications
- Experience supporting AI/ML or GenAI validation initiatives in pharmaceutical, biotechnology, clinical research, or other regulated life sciences environments.
- Experience working with cross-functional AI Engineering, Quality Assurance, Business, and CSV teams.
- Strong technical documentation, stakeholder coordination, and risk communication skills.
- Ability to work independently and manage multiple validation deliverables in a contract engagement.
📌 AI CSV / CSA Validation Analyst (México)
🏢 Programming.Com
📍 México