Queensland Health Artificial Intelligence Framework S6: Stage 6, ongoing monitoring and evaluation (Gate 5): establish KPIs aligned with the guiding principles covering quality (such as precision, recall and number needed to test), availability, responsiveness, data quality, drift and outliers, and fairness, inclusion and non-discrimination for every solution, with minimum standards at warning and critical levels; define retraining triggers, implement alerts to the system owner when standards are breached, monitor proactively for degradation, drift and unintended consequences with reports to governance or the oversight committee, build user feedback and audit mechanisms, re-evaluate regularly against the Stage 1 and 2 benefits and report periodically; retrain or refine as required and update the AI solution registry; report adverse events for registered software as a medical device to the TGA; and define and obtain approval for discontinuing the solution without affecting service delivery, recording the cessation date in the registry

What does S6 require?

Stage 6 continuously monitors AI systems in production, identifies issues, gathers end user and consumer feedback and refines the solution, because the quality of AI outputs varies over time as the environment and the use change, producing drift that is addressed by changing the solution or retraining on more relevant data as part of continuous improvement. The initiative owner aligns evaluation with the benefits and outcomes defined in Stages 1 and 2 and establishes KPIs aligned with the guiding principles to track impact, including quality measures suited to the use case such as precision, recall and number needed to test, availability and responsiveness, data appropriateness through data quality, data drift and outlier detection, and, for every solution, measures of fairness, inclusion and non-discrimination; sets minimum standards at warning and critical levels if not already done; defines model retraining triggers (on demand, at a set frequency or dynamically on KPIs); implements alerts to the system owner when minimum standards are not met; monitors proactively for performance degradation, drift and unintended consequences, with reports made available to project governance or the system-wide oversight committee; develops feedback mechanisms for user experiences and suggestions; sets up audit mechanisms; and plans regular re-evaluations with periodic monitoring reports so committees know of emerging issues. Models are retrained or the solution refined as required and the AI solution registry updated through the update form. If the solution is a registered software as a medical device and an adverse event is identified, it is reported to the TGA under its adverse event guidance. Where the solution must be discontinued and a process without AI reinstated, the requirements to do so without affecting service delivery are defined, the decision maker approves, an exit strategy or return to market may be needed, and the cessation date is recorded in the Queensland Health AI solution registry. The gate decides whether the solution transitions to business-as-usual in its current form, ceases, or returns to an earlier stage for improvement or replacement.