The Kill Switch Is the Smoke Alarm, Not the Fire Code
Taking some questions Penny Crosman asked me and turning the smoke alarm comment into actions for bank boards
Last Wednesday, American Banker ran a piece by Penny Crosman on a Wolters Kluwer survey finding that 72% of U.S. bankers say their institutions are least prepared on AI model kill-switch protocols or regulatory reporting of AI failures. You can read it here. Penny asked me three questions and used several of my answers, but as is always the case with a 6-minute read, some of the most practical material didn’t make it in. Below is the full, lightly cleaned-up version of what I sent her — including the part I most wish every bank board would read: the five things every institution should be able to produce on demand.
Do kill switches matter?
Yes — but they’re table stakes, not a strategy. A kill switch is the smoke alarm, not the fire code. If a bank is debating whether it can turn a model off, it has already lost control of that model. The real question isn’t “can you kill it?” — it’s “would you know when to?” Most institutions deploying AI today have no tripwires defining what abnormal behavior even looks like, so the switch would never get pulled until the damage was done.
And here’s a caveat worth sitting with: in banking, the kill switch itself is a risk. If your AI is mid-stream in payments, fraud screening, or collections, yanking it offline without a fallback is its own operational incident. A mature program has graduated controls — throttle, constrain, human-in-the-loop, then full stop — and never a single red button.
Could agentic AI without one lead to disaster?
Agentic AI changes the math completely. A traditional model may make a bad prediction; an agent may take a bad action — and then take a thousand more in 20 seconds before anyone notices. We’ve already seen what runaway automation does in markets: the 2010 Flash Crash, and Knight Capital, which destroyed itself in 45 minutes of unsupervised algorithmic trading. Now imagine that failure mode in collections or credit decisioning, where the victims are consumers instead of market makers. An agent misclassifying accounts and initiating collections actions at machine speed isn’t a hypothetical — it’s the obvious next headline. The Wolters Kluwer report is right to flag collections as the highest-risk function; that’s exactly where speed, autonomy, and vulnerable customers intersect.
But the worst case isn’t one catastrophic moment. It’s the quiet disaster: a drifting model making thousands of slightly-wrong, discriminatory, or non-compliant decisions per day, for months, invisible because nobody instrumented it. That’s not a kill-switch failure — that’s a monitoring failure, and it ends in consent orders, restitution, and a referral to DOJ, not just a bad news cycle.
What should minimum regulatory reporting look like?
None of this should be exotic — it’s SR 11-7 extended to systems that act instead of just predict. At minimum, every bank should be able to produce, on demand:
A complete model inventory — every AI system in production, including the “shadow AI” embedded in vendor products, which is where most of the unreported risk lives.
A decision audit trail — for any individual decision: what model, what version, what inputs, what confidence, who (if anyone) reviewed it. If you can’t reconstruct a decision, you can’t defend it to an examiner or a court. The closer the models are to mathematically deterministic, the better — at least until the models and their governing processes mature.
Drift and performance reporting against the validation baseline, with defined thresholds that trigger escalation.
An incident log — every time a model was overridden, throttled, or shut off, and why.
Third-party model attestation — banks own the risk of vendor AI whether or not they built it, and most can’t see inside it today.
The thing that worried me most at the FDIC
It wasn’t banks taking AI risk. It was banks taking AI risk they couldn’t see. That’s exactly what this survey is describing. The institutions that treat auditability as a design requirement rather than a retrofit are going to be the ones still scaling AI in five years. The rest will be explaining themselves to examiners.
And keep an eye on the agentic piece — that’s where I think the next enforcement cycle gets written.
The takeaway: 3 questions every bank board should ask
If you sit on a board, you don’t need to understand transformers or fine-tuning. You need to ask your CEO, your CTO, and every third-party tech vendor in the room three questions — and refuse to accept vague answers:
1. “Show me the complete inventory of every AI system touching our customers or our books — including what’s embedded in vendor products.” If the answer takes more than a week to produce, or comes back with the phrase “we’re still cataloging,” you’ve found your biggest risk. Shadow AI in vendor stacks is where the unreported exposure lives, and you own that risk whether you built the system or not.
2. “For any single decision one of these systems made yesterday, can you reconstruct it — what model, what version, what inputs, who reviewed it?” This is the difference between defensible and indefensible in front of an examiner or a court. If a vendor can’t answer this about their own product, that’s your answer about the vendor.
3. “How would we know — today, not in a quarterly review — that one of these systems is behaving abnormally, and what happens in the first hour after we find out?” You’re listening for defined thresholds, escalation paths, and graduated controls: throttle, constrain, human-in-the-loop, full stop. If the answer is “we’d turn it off,” go back and reread the part about the smoke alarm.
Three questions. None of them technical. All of them answerable on demand by any institution that has done the work — and unanswerable by the 72%.
Sultan Meghji is CEO of Frontier Foundry and the former Chief Innovation Officer of the FDIC.
Well done.