Before any system or model goes live it is evaluated and tested to a standard of reliability and safety that reflects the materiality of its use case. The FI pins down the main risks of the use case and sets clear, measurable thresholds; it reads the context of use into its reliability and safety assessments and borrows from recognised practice (IMDA's starter kit for testing LLM-based applications, for instance), and it checks performance across the plausible range from everyday conditions to edge cases. Three things are expected: evaluation measures that match the AI's objectives, with acceptable performance thresholds spelled out, written down and agreed between business owners, developers and reviewers; a toolkit of testing methods, among them out-of-sample and out-of-time tests, sensitivity and stability analysis across data distributions and time periods, sub-population analysis, stress tests including edge cases and, where fitting, adversarial tests, error analysis and benchmarking against alternatives, all on test data that reflects the FI's own context; and defences against overfitting, preferring the simpler model unless added complexity earns a clear performance gain, with regularisation, careful feature selection and robust validation such as cross-validation. Any risk or limitation surfaced during development is met with controls and guardrails before deployment (a guarded pilot is one way), and evaluation of newer technology such as Generative AI and agents targets its characteristic failure modes (hallucination, toxic or biased content, data leakage, susceptibility to adversarial attack) and checks that the guardrails actually hold.
The graph holds this control, the 0 it maps to, and the evidence behind each claim, over MCP and REST.