How to verify an AI-generated medical chronology in ten minutes

Ninety-nine percent of attorneys say they will not use AI-generated content they cannot verify. That is from Supio’s plaintiff-specific report, and it is the most sensible number in the whole survey.

What almost nobody has written down is how to actually verify a medical chronology once a machine has built it. So here is the procedure we use for checking one against the record, in the order it should be worked, and the one failure it will not catch.

Before any of this: can it be verified at all

Every entry has to cite the page it came from. Not the document, not the production — the page.

Without that, verification means finding each fact in the record yourself, which costs more than building the entry did. A summary you cannot trace is a summary you have to take on faith, and taking a thousand pages of somebody’s treatment history on faith is not a reasonable professional position.

Supio found 96% of attorneys very or extremely concerned about output lacking source attribution. It is also the failure behind every one of the AI sanction cases. That is the correct concern, and it is the first question to ask any vendor. Everything below assumes the answer is yes.

How to verify an AI medical chronology, in order

Work outside in. The order matters, because the early steps catch the failures that make the later steps pointless.

1. Check the boundaries — one minute

Find the first treatment date and the last. Confirm both against the record directly.

If the chronology starts three weeks after the date of loss and the record shows an emergency visit on day one, something upstream went wrong: a missing production, a set filed under a different name, an unreadable first page. Stop and fix that before checking anything else, because everything downstream is built on an incomplete set.

2. Check every flagged entry — three minutes

Anything the system flagged is load-bearing. A treatment gap, a billing discrepancy, a prior injury, an inconsistency between records. These are the entries you will argue from and the ones the other side will test.

Open each cited page and confirm the flag is what the record says. On a routine file there are rarely more than five or six.

Pay particular attention to gaps. A gap is an absence, and absences are inferred rather than read. Confirm no treatment actually occurred in the window rather than that no treatment was captured — those are different, and the second one is a records problem wearing a gap’s clothing.

3. Spot-check five ordinary entries — three minutes

Pick five at random from the middle of the chronology. Not the first five, which are the ones anybody would check, and not the interesting ones.

For each: open the cited page, confirm the date, the provider and the substance. You are checking that the ordinary machinery works, not looking for anything specific.

Five is enough. If two of five are wrong, the output is not usable and no amount of further checking will change that. If five of five are right, you have reasonable confidence in the bulk.

4. Reconcile the provider list — two minutes

This is the step people skip and the one that matters most. See the next section for why.

Take the list of providers the chronology identified and compare it against the billing index, the ledger, or the records custodian’s cover sheets — some independent list of who treated this person.

Every provider on that list should appear in the chronology. If one does not, you are missing a set.

5. Check the totals — one minute

Add the billed and paid figures and compare against the bills. These go straight into the demand package, and an error here is one you will have to correct in front of an adjuster.

The failure this will not catch

Everything above is good at finding entries that are wrong. It is bad at finding entries that are not there.

Omission is the real failure mode. A wrong date is on the page, and any spot-check finds it. A provider whose records were never in the production produces no entry at all — there is nothing to catch, no flag, no anomaly. The chronology looks complete because completeness is not something a document can show about itself.

This is why step four exists, and why it is worth more than the reading steps. Reconciling against an independent list is the only step that can detect an absence.

It is also why “the software missed nothing” is a claim nobody can honestly make, us included. What can be said is narrower and more useful: here is what it found, here is where each came from, and here is the list it was working from.

A wrong entry is a mistake. A missing one is a hole, and the only way to find a hole is to check against something outside the document.

What it reliably struggles with

Worth knowing before you test, so you test the right pages:

• Handwriting, which runs from perfectly readable to genuinely illegible. Any vendor claiming to handle all of it is overselling.
• Handwritten addenda on typed pages. The page looks clean, so it does not read as a hard page, and the annotation in the margin is the thing that mattered.
• Records filed out of order, which is common in large productions and makes date sequencing harder than it sounds.
• Duplicated productions, where the same visit appears twice from two custodians and is not obviously the same visit.
• Amended records, where an original and a correction both appear and only one is current.

None of these are reasons not to use the tooling. They are the reasons a person still reads the flagged pages.

What good looks like

After ten minutes you should be able to say four things:

1. The chronology covers the full treatment window, confirmed against the record.
2. Every flagged item is what the record says it is.
3. Five random entries checked out.
4. Every provider on the independent list appears.

That is a defensible position. It is not certainty, and anyone offering certainty on a thousand-page file is selling something. It is the same standard you would apply to work done by a paralegal you trust — spot-check the output, verify the load-bearing claims, reconcile against an independent source.

The difference is that this takes ten minutes instead of three days, and that the checking is now the job rather than the typing.

Why this is also the vendor test

The procedure above is a decent way to evaluate whether to buy anything at all. Run it on a sample of your own records during a trial, not on a demo file the vendor picked. Ours is open to drive without an email, which is the least a vendor can do.

If verification takes as long as building the chronology would have, the tool has saved nothing and the arithmetic does not work. If entries cannot be traced to a page, verification is not possible and the rest of the conversation is moot.

And if a vendor cannot tell you what their system reliably misses, that is not because it misses nothing.