What We Found Re-Judging 23,000 Control Mappings

Two controls can share a subject and share nothing that matters. C5, the German cloud standard, has a criterion titled "Access to cloud customer data". Read the title and you would map every access restriction control in every framework to it. Read the requirement and it does not restrict access at all: it is a duty to notify the customer within 72 hours when their data is accessed. It had attracted restrict-access mappings from eight different frameworks. A title that reads like a familiar control is the most expensive kind of wrong, because every check based on counting passes it.

We had 72 released crosswalk pairs resting entirely on mappings produced by one automated path. We pulled all 72 off sale and re-judged every mapping behind them in context, against both control sets, with a second pass that tried to refute each surviving claim rather than confirm it. 23,003 mappings went in. 12,944 were refuted, a 56 percent error rate. Some of them were high confidence on pairs that were already sold, which means they had been counting toward coverage a customer had paid for.

They were topic similarity, not evidence transfer. The question a mapping has to answer is narrow: does the evidence you produce for control A actually satisfy the requirement in control B, such that an assessor would accept it. "Both are about access control" does not answer that. The single most common wrong pattern was the policy magnet: a control requiring you to issue and communicate a policy being satisfied by a control that implements the thing, or the reverse. Those two are about the same subject and neither one produces the other one's evidence. That pattern alone was 23 percent of the corrections in one framework.

Ask for the number in both directions. GDPR into NIST SP 800-53 lands at 4.3 percent: almost nothing in 800-53 produces evidence for lawful basis, consent, data subject rights or international transfer. NIST SP 800-53 into GDPR lands at 35 percent, because GDPR Article 32 is a security article and 800-53 has enormous depth for it. Same two catalogues, an eight-fold gap, and no symmetric similarity score can produce that. If a vendor gives you one number for a pair of frameworks rather than two, they have computed a resemblance rather than judged a transfer.

A crosswalk that never rejects anything is not being judged. Across the 595 pairs released today, 65,733 candidate mappings were examined and 15,723 were removed in review, about one in four. Every one of them is still in the graph with the reason it failed, readable through a free tool with no account. That is the check worth running on any coverage claim, ours included: if the rejected set is empty or unavailable, the number in front of you was never argued with.

It tracks how prescriptive the target is, and it is unflattering. Outcome-worded targets like HIPAA and NIST CSF land between 50 and 79 percent, because one control can satisfy a broadly worded outcome. Parameterised catalogues like PCI DSS, FedRAMP and 800-53 land between 15 and 33 percent, because they specify values that another framework simply does not set. Across all 595 released pairs the median is 37.7 percent and the range runs from 3.7 to 98.2. If every pair in a vendor's catalogue lands in a comfortable band, that band is an artefact of the method rather than a property of the standards.

The metered path that produced those mappings is retired. Coverage is now published only for pairs that have been signed off, and each pair states its own level of review on the report itself rather than in the small print. Roughly two thirds of attempted pairs are refused rather than shipped thin, and the refusals are the reusable part: assessment procedures are not the controls they assess, work role definitions are not controls, and privacy transfers into a security framework only where that framework carries a privacy-scoped criterion. ISO 27701 into a set of security targets scored zero, not a small number. Zero is a finding. A small number would have been a product.