What It Takes to Make Generative AI Fit for GMP

Contributed Commentary by Valentina Armiento and Pierre Bonzani, Genedata 

September 11, 2026 | AI is reshaping pharmaceutical manufacturing. We see it being used to automate tasks, uncover insights, and optimize quality. By analyzing complex, rich datasets, AI can predict deviations, fine-tune processes, and accelerate routine tasks and documentation, leading to faster production and higher-quality outcomes. 

Good Manufacturing Practice (GMP) environments are precise, traceable, and controlled. Yet as operations scale and complexity grow, maintaining GMP standards is increasingly difficult. This is where AI excels. Techniques such as machine learning and deep learning can identify patterns across vast operational datasets that would otherwise go unnoticed, enabling early anomaly detection and more informed decision-making. As manufacturers face increasing pressure to deliver faster while maintaining quality, AI is an essential part of the modern digital toolkit. 

Generative AI (GenAI) raises a different set of questions. Instead of producing a prediction or a classification, it generates new content, including summaries, draft text, and responses. That output can vary with prompts, context, and the inherent stochasticity of generation, and is further sensitive to model configuration and model version, making traceability and repeatability harder to demonstrate at the level GMP expects. The opportunity remains significant, particularly in documentation and knowledge workflows, but the path to using GenAI in GMP is not the same as the path for ML and deep learning. It requires a clearly bounded context of use, strong linkage back to source information, and human oversight that is designed into the process rather than added at the end. 

AI Challenges: Adaptive, Probabilistic, and Hard to Validate  

In our experience, the lack of transparency is one of the core challenges for using AI in GMP-regulated environments. Traditional GMP computer systems are deterministic, rule-based software with fixed behavior. In contrast, modern AI models, especially large language models (LLMs) and deep learning networks, are often adaptive and probabilistic. Their behavior can shift with retraining, and their decision-making logic is often opaque, functioning as a “black box” that cannot clearly explain how conclusions are reached. 

This lack of transparency is a sticking point for regulators. In GMP environments, every decision that affects product quality must be traceable and justified. If an AI system recommends an action but cannot explain its rationale, it risks violating core principles of data integrity and auditability. Recent regulatory drafts, such as the draft EU GMP Annex 22 on Artificial Intelligence, reflect this concern by restricting dynamic or generative models from critical operations. In this current draft, only static models with fixed outputs are permitted in validated workflows; adaptive systems must be confined to low-risk tasks under human oversight. Specifically, generative AI (the kind that underpins chatbots and LLMs) is not permitted in critical operations and can only be used in low-risk areas under strict human-in-the-loop control. 

At the same time, this position is actively under discussion. In June-July 2026, the EMA GMP/GDP Inspectors Working Group is convening a multistakeholder workshop to gather expert input on whether, and under what guardrails, adaptive or generative AI models could be accommodated within a risk-based Annex 22 framework. A central theme will be the application of a quality risk management approach, specifically referencing ICH Q9(R1) principles, across the entire lifecycle of such AI systems, particularly in GMP high-risk areas where deficiencies may be difficult to detect but could have significant patient impact. This reflects an evolving regulatory dialogue rather than a finalized position. 

Unlike traditional software, AI systems depend on training data, model architecture, and versioning. Any update, whether retraining new data or switching to a newer model, can alter behavior, triggering the need for revalidation. This creates a dilemma: the more we update and improve the model, the more we risk disrupting its validated state and incurring a heavy revalidation burden. In regulated manufacturing, AI must not only perform well, but it must be able to defend its answers. 

Best Practices for Validating AI 

Trusting AI in production is challenging but not impossible. Its validation requires a shift from static, code-based validation to more dynamic, risk-based frameworks. The goal is to evolve legacy models to reflect the nature of probabilistic systems, without compromising traceability or control. Here are some best practices. 

1. Define Intended Use. Every AI system should have a clearly defined role, including the decisions it supports and the conditions under which it operates. For example, if a model is used to inspect vials, the validation protocol should specify the image types, resolution, and acceptance criteria. This ensures the model is tested on relevant data and used within its intended scope, an approach endorsed by EFPIA and other industry bodies.
   
2. Risk Assessment and Human Oversight. AI systems must be evaluated for the extent of autonomy they can safely have. In high-risk areas, such as quality release, AI should remain in an advisory role, with decisions confirmed by qualified personnel. This approach is broadly aligned with the human oversight principles set out in Regulation (EU) 2024/1689. Additionally, adaptive models should be locked after deployment and during operation or restricted from self-learning. 
   
3. Data Traceability. Every step of the AI process should be logged, including inputs, outputs, and user interactions. Model and training data versioning should be documented and maintained under change control. This mirrors GMP’s chain-of-custody principles and ensures that any flagged result can be traced back to its source.
   
4. Explainable AI (XAI). XAI is gaining traction to make complex models more transparent and understandable. Techniques such as confidence scoring or feature attribution help clarify how decisions are made. Regulators increasingly expect AI systems to provide human-readable justifications, especially when outputs influence GMP-relevant actions.
   
5. Integration. To be compliant, AI must integrate with existing quality systems. This includes applying change control, user requirements, and appropriate verification and qualification activities. Model updates should trigger formal review and documentation, as well as re-verification and re-validation where applicable, based on risk assessment, just like any other controlled change. Mapping AI processes to familiar computerized system validation (CSV) elements, such as design qualification or installation qualification, helps embed them into the compliance framework.
   
6. Cross-functional Governance. Successful AI validation requires collaboration across quality assurance (QA), IT, data science, and subject-matter experts. Many firms now establish AI governance boards to oversee deployment and ensure alignment across disciplines. This helps ensure that both technical performance and regulatory impact are considered from the outset. 

These strategies do not eliminate the complexity of AI validation, but they offer a practical way forward. By combining technical safeguards with procedural controls, manufacturers can confidently integrate generative AI into GMP environments. 

Sequencing Smarter: AI in GMP-Validated NGS Workflows