AI Validation

What is AI validation in regulated environments

Artificial intelligence is transforming the pharmaceutical industry, medical device manufacturing, and other regulated industries. Machine learning models improve defect detection on production lines, predict failures of critical equipment, assist in diagnosis, and optimize quality control processes.

However, deploying these systems in GxP environments without rigorous validation represents an unacceptable regulatory and quality risk.

Unlike traditional deterministic software, AI systems introduce specific challenges: their behavior depends on training data, can degrade over time without code changes (model drift), and their internal logic may be difficult to explain to an inspector.

AI validation adapts the established principles of Computer Software Assurance (CSA) and GAMP 5 to these particularities.

Why it is necessary: GxP, 21 CFR Part 11 and EU AI Act

GxP and Computer Software Assurance (FDA): The FDA's CSA guidance establishes that the level of validation rigor must be proportional to the risk to product quality and patient safety. High-impact AI systems in quality or safety decisions require documented evidence that they function according to their intended use.

21 CFR Part 11: If the AI system generates electronic records or signatures that replace paper records with regulatory relevance, Part 11 requirements apply:

• Access controls
• Audit trails
• Data integrity
• System validation

EU AI Act: The European regulation classifies AI systems according to their risk level. Systems used in medical devices, critical infrastructure, or decisions with significant impact on people are considered high risk and are subject to mandatory conformity assessment requirements, technical documentation, human oversight, and risk management prior to commercialization.

Our approach and methodology

Our methodology adapts the CSA/GAMP 5 framework to the specific characteristics of AI systems:

1. Intended use definition and risk classification

We precisely define what decisions the system makes or supports, what the consequences of an error are, and what level of human oversight exists. This classification determines the rigor of subsequent validation activities.

2. Model lifecycle documentation

We cover the entire lifecycle with complete traceability from source data to the model in production:

• Training data sources
• Preprocessing
• Model architecture and hyperparameters
• Training and cross-validation process
• Performance metrics

3. Performance validation and robustness testing

We design representative test sets independent of training. We evaluate performance under nominal conditions and in edge cases. We document the model's limits: which types of inputs produce reliable results and which do not.

4. Explainability and decision traceability

For high-risk systems, we implement explainability techniques (SHAP, LIME, feature importance analysis) and document how a model decision can be traced back to the input variables that determined it.

5. AI risk management

We follow the ISO 14971 risk management framework adapted to AI, identifying failure modes specific to ML systems:

• Bias in training data
• Degradation from data drift
• Unexpected behavior on distributions not seen during training

6. Post-deployment monitoring plan

Deployment does not end validation. We define performance indicators, alert thresholds, and retraining procedures within the change management system, ensuring that validation remains current throughout the system's lifecycle.

Use cases

ML models in diagnostics and quality control

Typical systems we validate:

• Computer vision for defect detection
• NIR/Raman spectrum classification models
• Automated image analysis in histopathology

We ensure that decision thresholds are justified, that performance on the validation set is representative of real-world use, and that a periodic model review process exists.

Automated decision systems in manufacturing

Typical systems:

• Process control systems based on predictive models
• Real-time parameter optimizers
• Anomaly detection systems in process data

We address validation from the perspective of impact on product quality and traceability of automatic changes to critical parameters.

AI software in medical devices (SaMD)

Medical devices with AI software components are subject to specific requirements from FDA (AI/ML-Based SaMD Action Plan), European MDR, and, where applicable, the EU AI Act.

We develop the required technical documentation, including the Predetermined Change Control Plan (PCCP).

AI tools for documentation review and compliance

NLP systems for:

• Automated batch record review
• Deviation detection in procedure text
• Trend analysis in complaints data

We validate both model performance and the associated access controls and audit trails.

Benefits: traceability, explainability and risk management

• Regulatory confidence: documentation that withstands scrutiny from FDA, EMA, AEMPS, and other agency inspectors, with objective evidence of the system functioning within its intended use limits.
• Complete traceability: from training data to production decisions, with records complying with ALCOA+ principles.
• Operational explainability: ability to justify model outputs to quality teams, regulators, and patients when necessary.
• Proactive risk management: early identification of biases, failure points, and degradation conditions before they impact quality or safety.
• System sustainability: monitoring and retraining processes that keep validation current over time, without the need for complete revalidation with every minor update.