Where this goes wrong

A weak answer is rarely a wrong one. It is usually a true statement doing less work than it appears to — a policy that exists but binds nobody, an artefact produced once and never since, a judgement recorded without the person who made it or the basis they made it on. Each entry below names the specific failure for its provision; the obligation page it links to sets out what discharges the duty instead.

The answer that does not hold

• IIA Standard 13.3Engagement Objectives and Scope

  A scope naming a technology instead of a system. "AI in the customer operations division" reads like a boundary and works as none: it does not say which models, which versions, which decisions, or what happens at the edges. The team then discovers its own scope during fieldwork, which is why the engagement overruns, and the report covers whatever it happened to reach. Nobody can tell afterwards whether a system was examined and passed or was never opened at all.

• IIA Standard 13.4Evaluation Criteria

  Criteria assembled after the fieldwork to fit what was found. The team tests, forms a view, then reaches for a framework that supports it — usually one nobody in the organisation had heard of before the report. It is rarely dishonest and always visible: the yardstick appears in the report rather than in the planning file, no one signed it off, and management’s first move is to dispute the source instead of the facts. Once that argument starts, the condition stops being discussed at all.

• IIA Standard 14.1Gathering Information for Analyses and Evaluation

  Volume offered as an answer to a reproducibility question. The file holds two hundred sampled outputs, every one reviewed and passed, and a memorandum explaining why two hundred was statistically defensible. Nothing in it says which system produced them or how anyone would obtain them again, so the sample size is a precise answer to a question the standard does not ask. Functions that fail here are rarely careless: they have applied a discipline built for records that sit still to a system that does not.

• IIA Standard 14.6Engagement Documentation

  The workflow tool mistaken for the file. Fieldwork lives in a GRC platform — a task per step, a status, an attachment or two, a reviewer who clicked approve — and the function reasonably assumes that is the record. What the platform does not capture is most of what the prescribed format asks for: the population as a definition somebody can run again, the source as an object rather than a name, and a review that recorded a question. The work was probably done well. It can no longer be shown to have been.

• IIA Standards 14.2 and 14.3Analyses and Potential Engagement Findings; Evaluation of Findings

  The adjective standing in for the analysis. A report says the model is biased, opaque or insufficiently governed, sets out the condition at length, and moves straight to a recommendation. Nothing states what the system was required to do, nothing reaches a control that failed, and the significance is a colour. It survives the closing meeting because everyone in the room already thinks the situation unsatisfactory — and it cannot survive anyone who does not, which is the reader a finding is written for.

• NIST AI RMF MEASURE 2.3 · IIA Standard 14.1AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment setting(s)

  A sample size lifted from a controls-testing table. Twenty-five items for a high-frequency automated control is a defensible number for a rule that behaves identically every time it fires, which is the assumption the table rests on. A model does not: it is one control operating a hundred thousand times with different behaviour at the margins, and the table has no view about that. The figure then gets cited in the paper as though it had settled the question of coverage, and it is the one number in the engagement nobody thinks to challenge.