GAMP 5 - Good Automated Manufacturing Practice D11: Appendix D11 Artificial Intelligence and Machine Learning

Machine learning sub-systems embedded in wider IT systems follow a risk-based life cycle framework aligned with the GAMP phases, with data management starting in the concept phase because data is the key resource, performance metrics acting as the technical specification of model acceptability, data managed with the same care as code, and attention to human factors, cybersecurity and legal liability, drawing on the FDA, Health Canada and MHRA good machine learning practice principles. Conventional life cycle work on the user interface, reports, security, access control and data integrity still apply, while the evolving outputs of an operational model need version control, monitoring, and change and configuration management. During concept, the business need and the workflow are set out, candidate algorithms researched or purchased with evidence of their development standards, constraints such as data availability, hardware, liability, regulatory and intellectual property factors determined, five principles of transparency, reproducibility, interpretability, applicability and liability considered, a dataset built to the objective with assumptions identified under a data governance strategy aware of data size, quality and prevalence, data acquired internally or externally with rights of use and personal data protection addressed, selected for diversity and freedom from bias, partitioned into training, validation and test sets with the test set excluded from all training and tuning, and prepared by profiling, cleansing, anonymizing, augmenting and labelling by domain-expert data scientists with documented evidence, and transformed. In the project phase model requirements including nonfunctional constraints and performance metrics against a gold standard are defined, a model type and architecture selected against operational, performance and regulatory requirements, model and data engineering infrastructure built with recorded trials, models trained with hyperparameter tuning that is not left entirely to automation, evaluated on the validation set with attention to bias from unrepresentative data and qualitative review of results, integrated by isolating the inference module from experimental code and specifying the interface, and verified, accepted and released under a process that defines whether training is part of verification and uses the held-out test data within user acceptance testing. In operation the sub-system is continuously monitored and evaluated, static systems change through change management with risk-based evaluation and robust testing, dynamic systems have metrics, key performance indicators and tolerance limits, additional training data is integrated systematically with staged evaluation, and performance is trended against quality control standards through the model's usable life. Supporting processes are risk management that adds the risks of machine learning intervention, dataset scope and labelling, retained knowledge, model maturity, production data changes and downtime, distinguishes static supervised systems handled by standard change control from dynamic or unsupervised systems needing continual performance evaluation, and reviews periodically for bias and overfitting; data governance that determines additional data needs and matches ground-truth format to the metric; and change management with defined processes for retraining triggered by performance requirements, additional inputs or changed intended use.