The evidence file

22 artefacts appear across MAS and AI in financial institutions. 0 of them are demanded by more than one obligation, which makes them the ones worth building first: the same document, properly made, discharges several duties at once. An artefact that is merely present does not discharge anything, so each entry records what makes it sufficient — that sentence is the part an assessor is testing.

What you will be asked for

• A data-flow map for one use case, caches included

  One obligation

  • Notice FSM-N05, paragraph 9 — Notice on Technology Risk Management

    Every place the information comes to rest, taken past the application into caches, logs, temporary files and the provider’s side of the boundary. The interesting entries are always the ones nobody designed.

• A deletion request executed to completion, indexes included

  One obligation

  • Notice FSM-N05, paragraph 9 — Notice on Technology Risk Management

    Traced end to end through source system, cache, index, log and any derived artefact. This is the test that separates a data map somebody drew from one somebody used, and very little passes it first time.

• A finding the development team did not simply accept

  One obligation

  • AI Model Risk Management §6.4 — Observations from a Thematic Review

    The disagreement, the remediation proposed, and the record of the reviewer agreeing it before the system was deployed. A history with no contested finding evidences either a very good team or a very quiet one.

• A re-rating triggered by a change in the system

  One obligation

  • AI Model Risk Management §5.3 — Observations from a Thematic Review

    One case where a rating moved because the system changed — autonomy raised, population widened, a human step removed — rather than because an annual cycle came round. It is the difference between a live control and a calendar entry.

• A step-independence test run against the deployed system

  One obligation

  • SAFR v1.0 — Safeguards for Agentic Finance at Runtime

    A multi-step workflow in which an early permitted action is shown not to authorise a later one. It takes an afternoon and returns a binary answer, which is why it is the first thing to ask for and the last thing most programmes have.

• Access evidence at artefact level, service accounts included

  One obligation

  • Notice FSM-N05, paragraph 9 — Notice on Technology Risk Management

    Who and what can read the vector store and the log, tested rather than described, with an explicit answer on whether the model’s own service credentials appear in the list.

• Envelope authentication evidence, not envelope storage

  One obligation

  • SAFR v1.0 — Safeguards for Agentic Finance at Runtime

    What signs the envelope at its origin, what verifies that signature before the disposition runs, and what the system does when verification fails. Keeping records is easy and settles nothing; the question is whether anything would reject a well-formed envelope from the wrong source.

• Escalation service levels with the decision recorded

  One obligation

  • SAFR v1.0 — Safeguards for Agentic Finance at Runtime

    The deadline for a human response, the format of what is put in front of that person, and the record of what they determined. These are the three things named as missing wherever escalation is a notification and a hope.

• One independent validation report with the stake declared

  One obligation

  • AI Model Risk Management §6.4 — Observations from a Thematic Review

    For a high-materiality system, stating the reviewer’s reporting line, their involvement in development, and their involvement in ongoing deployment on the face of the report. Independence asserted in a policy is not independence evidenced in a file.

• The action-type catalogue with a disposition assigned to each

  One obligation

  • SAFR v1.0 — Safeguards for Agentic Finance at Runtime

    Every action the agent can take, resolved to Deny, Escalate, Auto-Execute or Observe, with reversibility, financial materiality, customer impact severity, regulatory sensitivity and novelty shown as inputs. A catalogue with no reversibility column is the fastest tell that thresholds were set by convenience.

• The derivation of each Deny control to a named source

  One obligation

  • SAFR v1.0 — Safeguards for Agentic Finance at Runtime

    Each hard constraint traced to the policy, product rule, client mandate or in-force obligation it comes from. Where the rulebook holds a category heading and no populated list beneath it, the constraint exists in the diagram only.

• The distribution of ratings across the AI population

  One obligation

  • AI Model Risk Management §5.3 — Observations from a Thematic Review

    A count by band across the whole estate. Where nearly everything sits in one band the method has not discriminated, and the shape of the distribution says so before anyone reads the rubric.

• The exception log for systems deployed without their tier’s review

  One obligation

  • AI Model Risk Management §6.4 — Observations from a Thematic Review

    Which systems went live without what their rating required, who approved that, and what compensating control was applied instead. Exceptions are where a standard is shown to be real or decorative.

• The mandate register, with the point that enforces it

  One obligation

  • SAFR v1.0 — Safeguards for Agentic Finance at Runtime

    Scope, limits, conditions and validity period per agent, plus the component that refuses anything outside them. Authority here is explicit and structured, so a mandate held only in a system prompt is a suggestion the model may reason its way past.

• The mapping from band to control intensity

  One obligation

  • AI Model Risk Management §5.3 — Observations from a Thematic Review

    What each band actually buys: validation tier, monitoring frequency, documentation standard, approval level. Without it the rating is a label, and the assessment cannot be said to calibrate anything at all.

• The methodology with all three dimensions as scored inputs

  One obligation

  • AI Model Risk Management §5.3 — Observations from a Thematic Review

    Written scoring rules rather than the standing judgement of the model risk team, with each of the three producing a score traceable into the rating. A dimension appearing in the policy and not in the calculation has not been implemented.

• The referability determination for each AI data artefact

  One obligation

  • Notice FSM-N05, paragraph 9 — Notice on Technology Risk Management

    Training corpus, fine-tuning set, embeddings, vector index, prompt and completion logs, evaluation sets: for each, whether it is referable to a named customer, who decided, and on what reasoning. One blanket answer covering all of them is a policy statement, not a determination.

• The retention position for prompt and completion logs

  One obligation

  • Notice FSM-N05, paragraph 9 — Notice on Technology Risk Management

    The period applied, reconciled to the period applied to the systems the information came from. A log outliving its source has created a second record with different rules and no owner.

• The review of the measures themselves, not the ratings

  One obligation

  • AI Model Risk Management §5.3 — Observations from a Thematic Review

    What changed in the scoring measures, when, and why — kept distinct from the annual re-rating of systems. Most firms can produce the second immediately and have never performed the first.

• The third-party disclosure record for a model provider

  One obligation

  • Notice FSM-N05, paragraph 9 — Notice on Technology Risk Management

    What leaves the bank, under which contractual term, into which jurisdiction, and whether it may be used for the provider’s own purposes. The contractual answer and the observed traffic should be checked against each other.

• The validation standard, with the tier boundary written down

  One obligation

  • AI Model Risk Management §6.4 — Observations from a Thematic Review

    Which tier applies at which materiality, and what each tier requires of the reviewer. An unwritten boundary is settled case by case by whoever has capacity that month, which is the opposite of a standard.

• Two assessments differing only in autonomy

  One obligation

  • AI Model Risk Management §5.3 — Observations from a Thematic Review

    Comparable use cases with the first two dimensions held constant and the human reviewer present in one and absent in the other, showing the rating move. It is the shortest available demonstration that the third dimension does work.