The Exception Economy: Why Insurance Document Processing Will Never Be Fully Autonomous
By Brandon Robinson
TL;DR
- Every vendor sells AI as a clock — buy it, install it, set it on the shelf. AI is actually a plant.
- Three kinds of drift quietly degrade accuracy: appetite drift, broker drift, and model drift. All three compound after go-live.
- Plants die quietly. So does AI accuracy. You don't notice until it's already in your expense ratio.
- Ask one question before signing: whose job is it to water the AI? If you can't name them, you're already drifting.
What Is AI Model Drift?
AI Model Drift
AI model drift is the gradual degradation of an AI system's accuracy over time, even when the system itself hasn't changed. It happens because the world the model was trained on shifts — broker behaviors change, underwriting appetite evolves, document formats update, and the model's predictions slowly lose calibration against the new reality.
Most carriers think of AI as software they install. Drift is the reason that mental model fails. The system you bought in March is not the system you have in December — not because anyone changed it, but because the world it was trained on did.
The Clock Pitch
Every AI vendor demo ends the same way. There is a slide. The slide promises an eight-week deployment, a 99% accuracy benchmark, and a smooth on-ramp to production. The subtext is always the same: buy this once, install it, and it runs.
The mental model is that AI is a software install. You pay the license. You write the integration. You sign off on go-live. The system runs in production. The vendor's job is mostly done; your job is mostly done; the work is mostly done.
It is the wrong mental model. And the budget that flows from it is wrong. Carriers that approach AI as a software purchase consistently underbudget the work that determines whether the AI actually performs in year two, year three, year five.
AI is not software. AI is an operations capability. And operations capabilities have ongoing cost.
The Plant Reality
After go-live, drift starts immediately. Brokers begin to reword subject lines. Your underwriting appetite shifts a quarter point. A document type you rarely saw last year arrives more often. None of these shifts trigger an alert. None of them are dramatic enough for anyone to notice in the moment. But each one nudges the model's accuracy down a fraction of a point.
This is the work plants do. They drift quietly. A houseplant that needs water does not announce its degradation — the leaves yellow over weeks. By the time the change is visible, the plant has already been stressed for a long time. AI accuracy works the same way.
"It's like a house plant or a pet, and you have to continuously maintain it, feed it, deal with drift in appetite or drift in brokers and things like that."
— Jasper Li, NAMIC Webinar, May 2026
The dashboard will keep reporting the same headline metric. The dashboard is the wrong place to look. Drift shows up in the gaps between the dashboard's confidence intervals and what the underwriters actually do with the model's output — the silent corrections, the slow erosion of trust, the cases where the model "got it right" by the dashboard's definition but wrong by the business's. By the time drift is visible in your expense ratio or quote-to-bind, it has been underway for months.
Plants die quietly. So does AI accuracy.
The Three Drifts
Drift is not one phenomenon. It is three distinct forces, each with a different root cause and a different mitigation. Carriers that confuse them end up retraining the wrong thing.
Appetite drift
Your underwriting appetite shifts as the market hardens or softens, as you expand into a new line, as your risk leadership changes. The model was trained when "good risk" looked one way. Now good risk looks slightly different. The model still confidently classifies — wrong.
A concrete example: a model trained in 2024 to flag SOV inconsistencies in habitational property starts misclassifying when the carrier expands into mixed-use commercial in 2026. The new business looks superficially similar to the model. The carrier's actual appetite has moved. The model does not know.
Broker drift
Brokers do not follow your conventions; they follow their own evolving habits. Subject lines get reworded. Submission packets reorder documents. New brokers introduce new patterns. The intake model that was trained on the patterns of your existing broker network breaks the moment your top wholesaler hires a new junior who reformats every email differently.
This is the most operationally visible of the three drifts because it is the one that throws hard errors — submissions that fail to parse, classifications that miss. Broker drift is the canary.
Model drift
The most technical of the three, and the most universal. Even with stable appetite and stable broker behavior, ML models lose calibration over time as the underlying data distribution shifts. PDF formats evolve. Insurance vocabulary evolves. The model that was 99% accurate at deploy is 92% accurate eighteen months later because the input distribution has moved out from under it.
Every ML system is subject to this. It is not a vendor failure. It is a property of how supervised learning works in the real world.
Whose Job Is It to Water the AI?
Every office has plants that thrive and plants that die. The ones that thrive are the ones Sarah waters. The ones that die are the ones that are "everyone's job" — meaning nobody owns them.
AI is the same. In most carrier organizations today:
- IT thinks it is the vendor's job
- The business thinks it is IT's job
- The vendor thinks they are scope-limited to the contract
- Nobody owns drift detection
- Nobody owns retraining cadence
- Nobody owns vendor escalation when accuracy slips
If you cannot name the person responsible for AI maintenance — by name, not by team, not by role — you are already drifting. The role often does not formally exist in carrier orgs today. The skill set that supports it (model monitoring, MLOps, applied ML) is genuinely scarce, and the talent market for it is moving fast. Most carriers will end up buying that capability as a professional service from their vendor — but only if they budgeted for it upfront.
The carriers that catch drift early are the carriers that named a specific person, by name, before go-live.
The TCO math on AI is consistently broken in the same way. What gets budgeted on the original PO:
- Software license or subscription
- Integration cost (one-time)
- Internal change-management effort
What gets missed:
- Ongoing model monitoring
- Periodic retraining (vendor or in-house)
- Vendor professional services tail (often quarterly tune-ups)
- Internal MLOps staffing (if you're not buying that as a service)
What the industry data says
Annual AI maintenance runs 15–30% of initial deployment cost, every year — covering monitoring, retraining, performance tuning, and the human work behind all of it. Independent TCO breakdowns put 40–60% of total AI TCO post-launch — i.e., maintenance is typically the larger share of total ownership cost, not a rounding error. And the forecasting failure is documented: 56% of organizations miss their AI cost forecasts by 11–25%; nearly one in four miss by more than 50%. Care-and-feeding is consistently the line item they miss.
This is the line item nobody budgets for and everyone pays — either as a professional services bill they did not plan for, or as degraded accuracy that shows up downstream in expense ratio, quote-to-bind, and loss ratio.
"You're either going to have a professional services bill you weren't budgeting for, or you're just going to find your AIs get worse and worse over time."
— Jasper Li, NAMIC Webinar, May 2026
How to Budget for a Plant, Not a Clock
The fix is not complicated. It is just consistently overlooked. Four practical steps.
- Bake care-and-feeding into the original TCO. Add 20% annually to your three-year projection from day one. Negotiate the ongoing-services line into the original contract. Do not accept "we'll talk about that later."
- Name the owner. Before go-live, identify the human being whose job it is to water the AI. By name. Not by team. Not by role. Build their responsibilities into the deployment plan.
- Define drift signals. What metrics will tell you the AI is degrading before it shows up in expense ratio? Set the thresholds. Set the cadence. Have a runbook for what happens when a threshold is crossed.
- Plan for vendor portability. If the vendor goes away, gets acquired, or becomes too expensive — can you migrate? The answer should be yes by design, not by accident.
The carriers winning at AI in 2026 are not the ones who picked the best vendor. They are the ones who treated AI as the ongoing operations capability it actually is — staffed for it, budgeted for it, and named a person whose job it is to water it.
The carriers losing at AI are not the ones who picked the wrong vendor. They are the ones who treated AI as a clock — and then discovered, somewhere between year two and year five, that the clock had stopped keeping time.
Key Takeaways
1
AI is a plant, not a clock. Treat it as ongoing operations, not a software purchase. The budget shape that follows is materially different.
2
Three drifts will degrade accuracy: appetite drift, broker drift, model drift. Plan for all three from day one — each has a different root cause and a different mitigation.
3
If you cannot name who waters the AI, you are already drifting. Before go-live, name the person — by name, not by team. Without that, the ownership question becomes the failure mode.
Buying or evaluating AI for your operation? Make sure the vendor is selling you a plant, not a clock. Book a demo with SortSpoke →
Related reading: Why 95% of AI Pilots Fail covers the upstream version of this problem — the pilots that never make it to production in the first place. 9 Questions Before Buying Underwriting AI includes the vendor-evaluation questions that surface drift readiness. And Why the Best Insurance AI Keeps Humans in the Driver's Seat explains why human-in-the-loop is the operational discipline that catches drift before it shows up in your numbers.
Commercial P&C Insurers Guide to Solving the Underwriting Bottleneck
