Where this goes wrong

A weak answer is rarely a wrong one. It is usually a true statement doing less work than it appears to — a policy that exists but binds nobody, an artefact produced once and never since, a judgement recorded without the person who made it or the basis they made it on. Each entry below names the specific failure for its provision; the obligation page it links to sets out what discharges the duty instead.

The answer that does not hold

• Annex VII, points 4.3 to 4.5Conformity based on an assessment of the quality management system and an assessment of the technical documentation

  A data room offered as data access. The provider stands up a portal holding model cards, dataset documentation, evaluation reports and a schema, gives the assessor a read-only account, and treats the obligation as satisfied. Everything in the portal is a statement about the underlying material made by the party being assessed. The provision is drafted for exactly the situation the provider is invoking — material that cannot simply be handed over — and answers it with a technical route rather than a substitute artefact. One arrangement lets the assessor check; the other asks it to believe.

• IIA Standard 13.3Engagement Objectives and Scope

  A scope naming a technology instead of a system. "AI in the customer operations division" reads like a boundary and works as none: it does not say which models, which versions, which decisions, or what happens at the edges. The team then discovers its own scope during fieldwork, which is why the engagement overruns, and the report covers whatever it happened to reach. Nobody can tell afterwards whether a system was examined and passed or was never opened at all.

• IIA Standard 13.4Evaluation Criteria

  Criteria assembled after the fieldwork to fit what was found. The team tests, forms a view, then reaches for a framework that supports it — usually one nobody in the organisation had heard of before the report. It is rarely dishonest and always visible: the yardstick appears in the report rather than in the planning file, no one signed it off, and management’s first move is to dispute the source instead of the facts. Once that argument starts, the condition stops being discussed at all.

• IIA Standard 14.1Gathering Information for Analyses and Evaluation

  Volume offered as an answer to a reproducibility question. The file holds two hundred sampled outputs, every one reviewed and passed, and a memorandum explaining why two hundred was statistically defensible. Nothing in it says which system produced them or how anyone would obtain them again, so the sample size is a precise answer to a question the standard does not ask. Functions that fail here are rarely careless: they have applied a discipline built for records that sit still to a system that does not.

• IIA Standard 14.6Engagement Documentation

  The workflow tool mistaken for the file. Fieldwork lives in a GRC platform — a task per step, a status, an attachment or two, a reviewer who clicked approve — and the function reasonably assumes that is the record. What the platform does not capture is most of what the prescribed format asks for: the population as a definition somebody can run again, the source as an object rather than a name, and a review that recorded a question. The work was probably done well. It can no longer be shown to have been.

• IIA Standards 14.2 and 14.3Analyses and Potential Engagement Findings; Evaluation of Findings

  The adjective standing in for the analysis. A report says the model is biased, opaque or insufficiently governed, sets out the condition at length, and moves straight to a recommendation. Nothing states what the system was required to do, nothing reaches a control that failed, and the significance is a colour. It survives the closing meeting because everyone in the room already thinks the situation unsatisfactory — and it cannot survive anyone who does not, which is the reader a finding is written for.

• ISO 19011:2026 clause 7 · ISO/IEC 42006:2025 clause 7.1Competence and evaluation of auditors · Competence of personnel

  A training log offered as a competence file. Everyone on the team completed the same introductory module, the completion percentage is reported to the audit committee, and no document anywhere states what this engagement actually required of the people staffed on it. The tell is that the log would look identical if the work had been a payroll review. Clause 7 asks a question about the work first and the people second, and a log answers it in the wrong order.

• ISO 19011:2026, clause 5Managing an audit programme

  One line in the annual plan reading “AI governance review — Q3”. It passes the audit committee, it produces an engagement, and it answers none of the questions clause 5 asks. Nobody has said what the programme is trying to establish about the AI estate over the cycle, what fraction of that estate a single engagement reaches, or what was left out. The failure is not the engagement, which may be perfectly good. It is that a programme was never written, so there is nothing against which the coverage of that engagement can be judged.

• ISO 19011:2026, clause 6Conducting an audit

  Fieldwork that started before access was settled. The plan was issued, the opening meeting held, and three weeks in the team discovers that inference logs are retained for thirty days, that the model in production is two versions ahead of the one documented, and that the vendor’s evaluation harness is not available to the client at all. Everything after that is negotiation dressed as testing. The standard puts a preparation step before the conducting step for exactly this reason, and on AI engagements it is the step most often treated as paperwork.

• ISO/IEC 42006:2025Information technology. Artificial intelligence. Requirements for bodies providing audit and certification of artificial intelligence management systems

  An impartiality policy that names no threats. It is a page of principles, signed by a director, asserting that decisions are free from commercial influence, and it would read identically at a body with a large advisory practice and at one with none. What clause 5.2 contemplates is a body that has looked at its own arrangements, found the threats in them, and applied something specific to each. A buyer who reads only the policy learns whether the body can write; a buyer who asks for the analysis learns whether it has looked.

• NIST AI RMF MEASURE 2.3 · IIA Standard 14.1AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment setting(s)

  A sample size lifted from a controls-testing table. Twenty-five items for a high-frequency automated control is a defensible number for a rule that behaves identically every time it fires, which is the assumption the table rests on. A model does not: it is one control operating a hundred thousand times with different behaviour at the margins, and the table has no view about that. The figure then gets cited in the paper as though it had settled the question of coverage, and it is the one number in the engagement nobody thinks to challenge.

• Standards 14.3, 15.1 and 15.2Evaluation of Findings · Final Engagement Communication · Confirming the Implementation of Recommendations or Action Plans

  A tracker in which every row is green. Closure is evidenced by the date management supplied and the ticket reference management supplied, and the function has attested to nothing it observed itself. This survives until someone asks the obvious question about an AI control — is it still doing what it was put in place to do — and the file holds no measurement taken after the remediation to answer with. The finding was closed on an assurance from the party the finding was about.