Why insurance AI pilots stall before production

Eleven months ago somebody stood in your conference room and pulled fourteen fields off an ACORD 125 in about four seconds. The room was impressed, and it was right to be. Nobody has mentioned that project since March.

The approach that produced the demo deserves to be stated at its strongest before anyone takes it apart. Buy or build a model. Point it at the documents you already receive: the applications, the statements of values, the loss runs, the schedules of vehicles. Hold out a stack of forms it has never seen. Count how many fields it gets right. Ship when the count is good enough.

That is a sound way to start. It asks the cheapest question first and it answers in a week rather than a quarter. If a model cannot read a broker's schedule scanned at 150 DPI, nothing downstream matters, and you want to find that out before you have designed a workflow around it. Accuracy on held-out documents is a real measure of a real thing.

Nor is the outcome unusual. Capgemini's World Property and Casualty Insurance Report 2026, published in May and built on responses from 344 senior insurance executives, puts 10% of property and casualty insurers at scaling AI as a core capability and 60% still in exploration or proof of concept. Most of that 60% is not a heap of failed experiments. It is a heap of experiments that answered their question and then had nowhere to go.

Accuracy was the only question the pilot ever asked.

Take a value the pilot gets right. Total insured value, lifted off a statement of values, correct to the dollar.

Now put it on the account. Intake has a field called TIV that a producer types into. The policy record does not have that field at all; it carries a building limit and a contents limit per location, and rating reads the sum of those. The portfolio view has a column called Total Exposure, added two years ago by someone answering a different question, which includes business interruption.

The model did not get anything wrong. There was nothing for it to be right about. Every write into that account is now a judgment call about which of three fields the value belongs in, and a judgment call is precisely the thing the project set out to remove.

So a person reviews each one. That is the correct response, and it is also the end of the business case, because reviewing a proposed value takes longer than typing it did.

Then it is March. A broker disputes the schedule you quoted against and asks what the total insured value on that account was on the day the quote went out. You can find today's value. You can probably establish that it changed. What you cannot produce in an afternoon is which system wrote it, from which page of which document, on whose authority, and what the row read before.

None of that is a model problem.

The NAIC adopted its model bulletin on insurers' use of AI systems on December 4, 2023. As of April 1, 2026, twenty-four states and the District of Columbia had adopted it, beginning with Alaska in February 2024 and taking in Illinois, Pennsylvania, Michigan and New Jersey along the way.

Be precise about what that is. It is a model bulletin, not a statute and not a test. Departments adopt it one at a time. Its own closing section says the goal "is not to prescribe specific practices or to prescribe specific documentation requirements", and it says repeatedly that existing unfair trade practice and unfair claims settlement law already applies whether a person or a model made the call. Anyone describing it as a new compliance regime is selling you something.

What it does is tell you what you will be asked for. Section 4 is the part worth reading twice: regardless of whether you have written the program down, in an investigation or market conduct action you can expect to be asked about the development, deployment and use of any AI system that touched a consumer decision. Inventories and descriptions of the models. Where the data came from and what condition it was in. Validation, testing, retesting, and evidence that you looked for drift. All of the same for anything you bought, because the bulletin draws no line between a system you built and one a vendor sold you.

One factor sits underneath the rest. When the bulletin describes how much control a use case needs, one of the five things it weighs is how far humans are involved in the final decision. Answering that means producing records, not architecture. If the answer to "who approved this" lives in a thread in someone's inbox, you do not have an answer.

This is not a corner case for four carriers. In the NAIC's own surveys by line of business, 88% of the 193 auto insurers who responded in 2022 said they use, plan to use or plan to explore AI or machine learning. The figure was 70% of 194 home insurers in 2023, 58% of 161 life companies in 2023, and 92% of the 93 health insurers who responded in 2025. The auto number is the oldest of the four and worth discounting accordingly; the health one is the freshest and the highest.

Four states sit outside that count with insurance-specific guidance of their own rather than an adoption of the bulletin: California, New York, Texas and Colorado. Colorado has gone furthest, with its own insurance regulation in force since November 2023 and amended again in October 2025. And the supervision is becoming more operational: twelve states were piloting an AI systems evaluation tool as of March 2026, with adoption expected at the NAIC's fall national meeting.

The interesting question was never whether a model can read the schedule. It is whether there is one place to put what it read, and whether the write leaves something behind that holds up in March. That is where the effort went.

So there is one mesh. Objects, attributes and relationships are defined once, in your own words, and extraction, rating, workflows, reporting and the assistant all resolve against those same definitions. The Data Mesh is the whole idea, named: nothing keeps its own copy, so total insured value is one field rather than three, and a correction lands once and is true everywhere it is read.

Changing that model is product definition rather than an implementation project. Add an object, an attribute or a relationship in the data model and it is usable in every feature immediately. Objects, attributes, views, document types and rate books version together and publish as one release, which means a rule you got wrong is a dated change you can point at rather than a deploy nobody logged.

Only then does the assistant become interesting. Ask Riska runs as the person asking, so it reaches the records that account reaches and nothing beyond them. Each lookup names itself as it runs and stays in the thread, so you can see which query produced a figure before you act on it. A record edit arrives as a preview showing every field it would set and anything that failed validation, and nothing is written until a person approves it. The audit row that results is attributed to the assistant and to the conversation it came from.

None of that makes the model read a schedule any better. It makes the answer something a person will act on, and it turns the paragraph a regulator asks for into a query rather than an excavation.