Your document pipeline should stop, not guess
- Written by
- Riska Underwriting TeamInsurance operations & underwriting
- Published
- Length
- 4 min read
The cost of a confident wrong answer
A submission arrives at 4:40pm with eleven attachments. Nine are the ones you expected. One is a page scanned at 150 DPI that could be anything. One is an ACORD 140 for a location the ACORD 125 never mentions.
Somewhere in the next two minutes a piece of software has to decide which product this submission belongs to, and it is not certain. Most intake pipelines resolve that by ranking the candidates and taking the top one. The run finishes. The record is created. Everything looks fine.
It looks fine until the quote comes back priced against the wrong program. By then the wrong answer has been copied onto a policy record, a document has been generated from it, and someone has sent that document to a broker. The two minutes you saved at intake cost a day to unwind, and the person who finds the error is the one you least wanted to find it.
The uncertainty was real. The problem is what the software did with it.
Eight stages, and where a run is allowed to stop
Every submission triage run in Riska is the same eight stages, named on the run so you can see exactly where one stopped.
Classify. Reads each file and decides what kind of document it is.
Product Classify. Per file, proposes which product it belongs to.
Reconcile. Takes those per-file proposals and resolves one product for the whole submission.
Extract. Pulls the configured fields out of each parsed document.
Merge. Combines the per-file values into one set.
Match Product. Resolves which existing record this submission belongs to, and stops rather than choosing when more than one candidate matches.
Changeset. Turns the merged values into a set of proposed writes.
Apply. Commits those writes to the record.
The last two are the ones this argument turns on. Changeset produces the writes. Apply performs them. They are separate stages because the gap between them is where a person can stand.
Two ways to handle an ambiguous submission
The difference is not whether the pipeline is uncertain. Both are. It is where the uncertainty surfaces, and who is holding it when it does.
| Question | A pipeline that guesses | A pipeline that stops |
|---|---|---|
| Reconcile cannot pick a product | Takes the highest-scoring candidate and continues | Parks the run and opens a review |
| When the error surfaces | At quote or at bind, days later | At intake, in minutes |
| Who finds it | The broker, or the audit | The person who already owns the submission |
| Cost to unwind | Reverse the record, reissue the documents, explain the call | Pick the right product, resume the run |
| What the audit trail holds | A value with no author | A named decision with a timestamp |
| Time saved on a clean submission | The same | The same. A clean run never stops |
Why the changeset is the boundary
Auto-apply is a setting on the pipeline, and it is off by default. With it off, a run halts after the Changeset stage. Every value the pipeline extracted is sitting in one reviewable object, each field carrying the document and page it came from, and none of it has touched the policy record.
That is a real boundary rather than a review queue bolted on after the fact. There is no partially written record to reconcile, no half-applied endorsement, nothing to reverse. Approving the changeset applies every operation together. Rejecting it leaves the record exactly as it was.
It also means the setting is yours to move. Turn auto-apply on for the document types where you have earned the confidence and leave it off for the ones you have not, so a clean renewal packet and a first submission from a new broker do not have to be treated the same way.
What happens when a run cannot finish a stage
- 01
The run parks where it stopped
Reconcile is the stage allowed to do this. When it cannot resolve a product from the per-file candidates, the submission moves to AWAITING_REVIEW instead of taking the top of the list.
- 02
A review opens with a task attached
The work lands on a task list someone already looks at rather than in a queue nobody owns. The review is idempotent per submission and stage, so a workflow retry cannot open a second copy of the same question.
- 03
One person makes one decision
The review asks only for the thing the pipeline could not determine, usually which product this is. Everything the run already worked out is still there and is not thrown away.
- 04
The run resumes from where it stopped
Resolving the review signals the parked workflow, which continues into extraction with the product it was given. The stages before it do not run again.
Restarting from the stage that was wrong
The other half of not guessing is being able to fix a run cheaply. A boundary that forces someone to look is only worth having if acting on what they saw is fast.
A run can be restarted from a chosen stage. Everything before it is rehydrated from the stored per-stage results rather than recomputed, and everything from that stage onward runs again. Correcting a product mapping does not mean re-parsing a fifty page loss run. Re-running extraction after a field mapping changes does not mean reclassifying eleven attachments.
That is what makes tuning document intelligence survivable. Change the mapping, restart from Extract, look at the changeset. The loop is minutes, and it costs one stage of compute rather than all eight.
None of this makes the pipeline more accurate. It makes being wrong cheap, which across a book of business is the more useful property.
Questions we get asked
- Does every submission need someone to approve it?
- No. Auto-apply is a per-pipeline setting, and with it on a run goes straight through to the record without stopping. It ships off because the first weeks of any intake configuration are the weeks you most want to see what the pipeline decided before it acts.
- What happens to the rest of a submission when one attachment will not classify?
- Nothing. The per-file stages aggregate rather than fail. A page that cannot be classified is reported as a failed file on that run and the other attachments continue through the pipeline. Only a total failure of every file fails the submission.
- Can I re-run just the extraction step?
- Yes. A run can be restarted from any stage. Everything before the chosen stage is rehydrated from its stored results and everything after it runs again, so re-extracting does not re-parse or reclassify the documents.
- How is this different from a confidence threshold?
- A threshold still produces an answer and attaches a number to it, which leaves a person deciding what to do with a value marked 0.71. Parking the run produces no answer at all until someone supplies one, and the decision is recorded as theirs.
- Where do I see what a run actually decided?
- Each stage writes a job record you can open. Classify, extract and the rest each list the file they ran against, their inputs and their outputs, and an extract job shows every field it pulled with the page it came from.
Keep reading
Introducing the architecture for agentic premium
Agents can do the high-context work of an underwriter at the speed of a machine, and carry that judgment across every account on the book. Most insurance software was built for a person who opens one file and finishes it. Today Riska is introducing a hybrid architecture engineered for people and agents in equal measure: the Data Mesh connecting every submission, policy, premium posting and claim, Ask Riska speaking SQL over the live book, and reporting and connectors that keep every motion on the same records.
Why the agentic era will break insurance systems
Insurance systems were built for a person who opens one account and finishes it before the next one starts. More product lines, and a broker's agent sending the same risk to more carriers, mean more submissions and more operations than that person can decide on. Anthropic hit the same wall with git: about 2,500 merges a day in May 2026, more than 30,000 by September, and the tool built for a smaller day was the bottleneck. The platform that cannot score, refer and answer while the submission is still open loses the account.
What a bordereau is, and why delegated authority still runs on one
A bordereau is the schedule of risks, premium or claims an MGA sends its carrier under delegated authority. It goes monthly or quarterly, inside a window after the period closes that the binder sets. That lag was a limit of the software, not of the business. When the bordereau is a view of live records, the carrier, the MGA and their agents read the same numbers without the month-end export. Live, not exported.
See it on your own submissions.
Bring a real ACORD form and a real appetite guide. We will run them through Riska on the call.