We measured it rather than asserting it. The same model answered the same 144 questions twice: once from its own knowledge, and once with the control graph. Every question is about a published standard, so both answers can be checked against the issued document.
The result below includes the part that does not flatter us.
It does not show that our model is better than someone else's. It is the same model on both sides. Testing one vendor against another would need paid API calls to those vendors, and that is a different experiment answering a question we are not in the business of answering.
It does not show the graph is always worth it. On the five standards everybody has memorised, the gap is 5.3%. If ISO 27001 and SOC 2 are all you work with, a good model already knows them and this buys you very little.
It does show what happens past the famous few. On a German cloud catalogue, an Australian information security manual, an Australian aged care standard, an EU digital identity regulation and a NIST supply chain publication, the unaided model dropped to 63.2% and admitted 28 gaps. With the graph it answered 98.9% correctly and admitted none.
144 questions of three shapes, drawn from the graph and verified against it. What is the title of control X. Which control has title Y. Does a control numbered Z exist, where 27 of the numbers asked about genuinely do not exist in the standard, so that answering yes to everything is a detectable failure rather than a free pass.
Two cohorts, same questions, same model. One was given no lookup tool and told to answer “unknown” rather than guess. The other read the control lists from the graph and answered offline from them. Grading ignores case, punctuation and any namespace prefix, so nothing is lost to formatting, and a title counts as correct when it shares at least 60% of the reference's content words.
Frameworks deliberately excluded: four were dropped after an earlier run because the graph holds more than one representation of them, or stores their control “numbers” as sentences. A question whose answer depends on which representation you happen to hit measures our own inconsistency rather than anyone's knowledge. Those are real defects on our side and they are being fixed separately.
The first version asked only about the five best known standards. The unaided cohort scored 100%, using two tool calls, which were reading the questions and writing the answers. It had not cheated; those standards are simply in every training corpus many times over. That result is kept as the control group above.
A later version generated its fake control numbers by checking them against a filtered subset rather than the whole framework, so it offered ISO 27001 8.15, which is Logging and entirely real, as a fabrication. Both cohorts answered correctly and both were marked wrong. Four such questions were found and every negative in the published set has since been verified absent from the graph individually.
The run before this one scored the unaided cohort at 144 out of 144. It had not cheated either: each control appeared in three questions, and the “what is the title of X” and “which control has title T” questions are inverses, so the paper disclosed its own answers. The cohort said so in its own report, which is the only reason it was not published. Every control now appears in exactly one question and that is checked before a run.
This is recorded because a benchmark that hides its own errors is asking for exactly the trust it claims to measure.
The unaided cohort produced 7 confident wrong answers, separate from the 28 it admitted it did not know. It stated that eIDAS 2.0 contains articles 43 and 47, which it does not, and misquoted the titles of SOC 2 CC5.2 and PI1.2, which are in every corpus. Those are the dangerous ones: a wrong control reference looks exactly like a right one until somebody checks it.
The graph-backed cohort got one question wrong out of 144, and it is a formatting artefact rather than a wrong answer: it returned 258 where the graph stores the zero-padded 0258. We left it scored as wrong. The grader applies one rule to both cohorts and adjusting it here would move the number in our own favour, which is not a thing this page should do.
The question set, both sets of answers and the grader are in the repository under scripts/benchmark/. Every question can be re-asked against the same free tools, which need no account:
curl -sG "https://api.theartofservice.com/api/agent/framework-controls" \ --data-urlencode "name=C5 (Germany)"Point your own agent at it
Run 2026-08-20. Re-run when the graph changes materially; the numbers on this page are read from the recorded result rather than typed in.