MAS and AI in financial institutions
AI Model Risk Management paper, section 6.5 — Deployment, Monitoring and Change Management
MAS’s December 2024 thematic review records what selected banks do once an AI model is live: pre-deployment checks, measures tracked against predefined thresholds, an issue process, contingency standards, and periodic review on top of ongoing monitoring. It is an information paper and obliges nobody. The Technology Risk Management Guidelines underneath it are a supervisory expectation, and a retrain is still a release.
§ 1Who it binds
Which systems and which firms
Financial institution
Any AI the firm’s own definition catches, once it is deployed. The paper’s observations were drawn from a thematic review of selected banks, and it extends their relevance to other financial institutions by encouragement rather than by application. Monitoring intensity follows the firm’s risk-materiality rating. The change and release management expectations in the Technology Risk Management Guidelines reach the full applicability list of those Guidelines rather than only the banks reviewed.
§ 2In practice
Watching inputs, and calling it monitoring
Two documents, two statuses, and the page turns on holding them apart. *Artificial Intelligence Model Risk Management — Observations from a Thematic Review* was published on 5 December 2024 as an information paper. Information papers sit outside MAS’s taxonomy of regulatory instruments altogether: they are publications, and this one describes itself as good practice MAS "encourages all financial institutions to reference". Nothing in section 6.5 is owed by anybody. Sitting beneath it are the Guidelines on Risk Management Practices – Technology Risk, and those are a Guideline in MAS’s own sense — contravening them is neither a criminal offence nor a civil penalty, but, in the words of the Guidelines themselves, "the degree of observance with the spirit of the Guidelines by an FI is an area of consideration by MAS". So the monitoring content is observed practice, the change and release content is a supervisory expectation, and a page that blurs the two will overstate one and understate the other.
Annex A of the paper supplies the vocabulary, and it is worth adopting rather than reinventing: data drift, concept drift — for which the paper names the Characteristic Stability Index as a common measure — and model drift as the broader term covering both, tracked through PSI and CSI and by monitoring the statistical characteristics of the model’s predictions. Section 6.5.4 records that measures were tracked against predefined thresholds. What that vocabulary does not contain is an outcome. PSI describes the input distribution; a flat PSI says the population arriving at the model looks like the population it was fitted on, and says nothing whatever about whether the answers are still right. A firm with a drift dashboard and no realised-accuracy measurement is monitoring the half that can be computed on the day, and the discipline for measuring the other half on a system whose output is probabilistic is set out under testing a probabilistic control.
Section 6.5.2 keeps non-AI pre-deployment checks in scope, and that sentence does more work than its length suggests. A model release is a software release. The Technology Risk Management Guidelines carry sections on change management (section 7.5) and software release management (section 7.6), and nothing in them exempts a retrain. In most firms, though, a code change and a model change travel through different governance: one through release management with approvals and a rollback plan, the other through a model-risk workflow that ends at revalidation and hands the artefact to an automated pipeline. A monthly retrain deployed under the heading of routine maintenance has bypassed the release control that a code change of identical consequence would have triggered, and the firm will usually be unable to say who authorised the version currently answering customers.
The last two sections are the ones most often collapsed. Section 6.5.4’s predefined thresholds and section 6.5.5’s process for reporting, tracking and resolving issues are two halves of one control, and the assessable question is never what the thresholds are — it is what happened the last time one was breached. Where the answer is that the breach was noted and the threshold later widened, the control is a dashboard. Section 6.5.6 adds contingency standards and processes, and section 6.5.7 adds periodic reviews *in addition to* ongoing monitoring. Those two ask different questions. Monitoring asks whether the model still behaves as it did; the periodic review asks whether it is still the right model for a business problem that may have moved underneath it. The Union analogue, a documented post-market monitoring plan under Article 72, is a statutory artefact rather than an observed practice, and the difference in status is exactly what a Singapore firm has to keep straight.
§ 3When it applies
Two documents, two dates, neither a deadline
18 January 2021
The date of the current edition of the Guidelines on Risk Management Practices – Technology Risk. Guidelines are supervisory expectations, so this is a publication date rather than a compliance deadline; no duty commenced on it.
5 December 2024
The publication date of the information paper Artificial Intelligence Model Risk Management — Observations from a Thematic Review. It records practice observed at selected banks and encourages reference to it. Nothing fell due on this date.
§ 4What discharges it
What the monitoring programme must produce
The artefacts an assessor asks to see, and what makes each one sufficient rather than merely present.
01
The monitoring specification for one live model
Each measure, its threshold, its frequency and its named owner, on one page. Where the owner column is a team rather than a person, nothing has been assigned and the breach will be everyone’s.
02
Realised performance against settled outcomes
Accuracy measured on ground truth once it is known, with the lag stated and the mature vintages reported separately. Without it the firm is watching inputs and inferring that the answers held.
03
One threshold breach traced through to closure
The alert, the action it triggered, the issue record, and the resolution — end to end, on a real event. A breach closed by widening the threshold is a finding rather than a resolution.
04
The change record for the most recent retrain
Which release controls were engaged, who approved the version now in production, and whether the technology-risk change management route was used or a model-specific workflow stood in for it.
05
An exercised contingency arrangement
What the firm does when a model has to be withdrawn, with proof it has been run rather than written: who ran it, how long the switch took, and what capacity the fallback actually sustained.
06
The most recent periodic review of continued fitness
Distinguishable on its face from the monitoring pack, and reaching the question of whether this remains the right model for the current business problem rather than whether it still behaves as before.
§ 5Worked example
Worked example — a green dashboard and a rebuilt scorecard
A retail bank runs a behavioural scorecard, refreshed by an automated pipeline on a rolling twelve-month window every month. A monitoring dashboard shows PSI on the top twenty features, refreshed nightly, and has been green all year. Delinquency outcomes are only observable nine months after a decision. The model was materially rebuilt in March, when the feature set changed, and the rebuild went out through the same monthly pipeline under the change class used for scheduled maintenance.
What does section 6.5 look for that the dashboard does not supply?
Three things, and none of them is a better drift metric. First, outcome measurement. Green PSI over a year of monthly refreshes is a statement about inputs; the bank cannot yet say what the March rebuild did to realised accuracy, and the honest position is a monitoring specification that states the nine-month lag, names the cohort whose outcomes will settle when, and reports the vintages already mature rather than waiting for all of them. Second, change control. A rebuild that changed the feature set is not the monthly refresh the maintenance class was written for, and the Technology Risk Management Guidelines’ change and release management expectations do not carve out a model. Whoever approved the March release, and against what, is the record that should exist and generally does not. Third, the distinction between monitoring and periodic review: nothing in the year’s dashboards asks whether a behavioural scorecard fitted on pre-rebuild data is still the right instrument for the portfolio the bank now has, and section 6.5.7 treats that as a separate exercise on its own cadence.
§ 6What a weak answer looks like
Green panels, no thresholds
A drift dashboard offered as the monitoring programme. It is well built, it refreshes nightly, and every panel is green. Nobody can name the threshold that would turn one red, nor point to an occasion when one did, nor produce the issue record that followed — and the panels all describe the data going in rather than the decisions coming out. The most recent retrain went live through an automated pipeline while the dashboard reported on the model it replaced, which nobody noticed for a fortnight.
§ 7Exposure
What actually follows
No criminal or civil penalty
The information paper is a publication and has no legal effect. The Technology Risk Management Guidelines are a Guideline: MAS states that contravening guidelines is not a criminal offence and does not attract civil penalties, but that specified institutions should observe their spirit and that how well an institution observes them may bear on MAS’s overall risk assessment of it. The practical consequence of weak monitoring is nearer to hand than either — a model that has stopped working is discovered by a customer, and a firm that cannot show when it stopped cannot answer for the interval.
§ 8Elsewhere
The same duty in other regimes
Where another instrument addresses the same obligation. These are correspondences, not comparisons — the Council does not rank one framework against another.
Article 72 requires a provider to operate a post-market monitoring system to a documented plan forming part of the technical documentation. It states a statutory artefact where section 6.5 records observed thresholds and cadences.
Article 15 addresses feedback loops in systems that continue to learn after deployment, which is the failure the drift vocabulary in Annex A is used to detect.
Testing a control whose output is probabilistic is the method question behind outcome measurement: what sample, what ground truth, and what conclusion the result supports.
MAS instruments
The draft Guidelines, paragraph 4.23, would have reasoning processes, actions taken and tools used monitored across workflows. It remains a proposal in a consultation paper.
A correspondence indicates that two instruments address the same underlying obligation. It is not a mapping endorsed by either body, not a statement that one satisfies the other, and not a judgement about which is more demanding.
§ 9provenance
The provision itself
This page sets out what the instrument requires and what discharges it. The official text is the authority — these go straight to it.
- Artificial Intelligence Model Risk Management — Observations from a Thematic Review, December 2024. Section 6.5 Deployment, Monitoring and Change Management, with the drift definitions in Annex A.
- Guidelines on Risk Management Practices – Technology Risk, 18 January 2021 — sections 7.5 Change Management and 7.6 Software Release Management, cited by number and title.
- MAS records the publication date of the information paper as 5 December 2024.
- The current edition of the Technology Risk Management Guidelines is dated 18 January 2021.
- MAS’s statement of the legal effect of Guidelines, and the absence of information papers from the instrument taxonomy.
§ 10Also read
Also read
Testing a probabilistic control
ISO/IEC 42001 clause 9.1 — monitoring and measurement
Certification
Assessed on the same standard of evidence
Every Council credential is examined on applied judgement against a published anchor, set out the way the obligations on this page are. The Certified Applied AI Associate, the Council’s flagship, is open to any adult today, its fee waived by a coupon code from the AI-Readiness Check, and the register lists what is open for enrolment.