Last updated: October 2026 · Reading time: about 12 minutes
What this guide covers
What AI medical record summarisation actually does
The task sounds simple: read hundreds of pages of medical records and produce a summary a paralegal would normally have taken days to build. In practice, four distinct operations sit inside that description, and each has different accuracy characteristics.
1. Extraction. Pulling structured data out of unstructured records — treatment dates, provider names, facility names, diagnoses, procedure codes, medications, billing amounts. This is the most reliable operation because the source text is usually explicit. A date on a discharge summary is a date.
2. Chronology building. Sequencing those events into a timeline. This is where errors begin, because it requires reconciling conflicting dates across records — a referral letter may state one first-visit date, the provider's own notes another, and the billing record a third.
3. Summarisation. Condensing the clinical narrative — what happened, what was found, what treatment followed, what the prognosis is. This is where hallucination risk sits, because the model is generating prose from source text rather than extracting fields from it.
4. Gap detection. Identifying periods with no treatment, diagnoses mentioned once but never followed up, or referrals that never resulted in a visit. Gaps in care are the single most common argument insurers use to reduce a claim, which makes this the highest-value output and the highest-risk one to get wrong.
The tools that work well in 2026 do all four. The tools that work badly do the first one and claim to do the other three.
The accuracy problem: why "95% accurate" is unverifiable
Almost every AI medical record tool will state an accuracy figure. Almost none of those figures mean what a firm thinks they mean.
Three problems with the claim:
- There is no ground truth. "95% accurate" against what baseline? A vendor's own test set — chosen and labelled by the vendor — proves the vendor can score well on their own test. It does not tell you how the tool performs on a specific case with a specific record set.
- Accuracy is not uniform across outputs. A tool may be 99% accurate on treatment dates and 70% accurate on prognosis. A single headline number hides that difference, and the 70% matters more in a demand package than the 99%.
- The errors that matter are not random. A random 5% error rate is tolerable if the errors are easy to spot. A model that systematically misattributes one provider's note to another, or that rounds "approximately three months" into "three months," produces errors that are plausible enough to survive review — and damaging when they surface in a deposition.
The traceability distinction
This is the concept the rest of this article turns on, and it is the thing most firms get wrong when they evaluate tools.
Accuracy and traceability are different properties, measured in different ways:
- Accuracy answers: how often is the output correct? It is a statistical property of a model against a test set. You cannot measure it on a single case, and you cannot verify it yourself. You have to trust the vendor.
- Traceability answers: can I see where each claim came from? It is a structural property of the output. You can check it on any single case, in any single summary, without any vendor involvement. You can verify it yourself in five minutes.
For litigation use, only traceability is checkable. And because it is checkable, it is also the only property that lets you defend a summary if it is challenged.
| Traceability level | What it means | Usable in litigation? |
|---|---|---|
| Black box | Summary is produced with no citations. The reviewer takes it on trust. | No. Not defensible in a deposition or filing. |
| Document-level | Each summary sentence lists which documents it came from, but not which page or passage. | No. Too coarse — a 40-page record set cannot be checked against a document list. |
| Source-linked | Each summary sentence links to a specific page and passage in the source record. | Yes. The reviewer can verify in one click. This is the minimum standard. |
| Source-linked with confidence scores | As above, plus each extraction carries a confidence score so low-confidence items are flagged automatically. | Yes — best practice. The reviewer knows where to look first. |
The practical consequence: when a vendor demonstrates their tool, ask them to show you the citation on a specific sentence. Click it. Does it open the actual page? Does the highlighted passage match the sentence? If the demo does not do this in one click, the tool is not traceable — regardless of what the slide deck says about accuracy.
What a usable summary must contain
Beyond traceability, a medical record summary produced for a personal injury case must cover six things. If any are missing, the summary is incomplete for its purpose.
| Element | Why it matters | Common AI failure |
|---|---|---|
| Chronology of treatment | Establishes causation and continuity | Misorders events when sources give conflicting dates |
| Providers and facilities | Establishes who treated what; supports expert testimony | Conflates provider names when a single record mentions several |
| Diagnoses with dates | Establishes injury profile and severity | Merges similar ICD codes; may state a diagnosis that appears only as a rule-out |
| Gaps in care | The single most common reduction argument; must be identified proactively | Misses short gaps; may not flag gaps caused by provider cancellations |
| Prognosis and permanency | Drives future medicals and general damages | Quotes hedged language ("may require") as definitive |
| Pre-existing conditions | Must be identified early; discoverable and damaging if missed | May not flag conditions mentioned incidentally |
The six elements above are the minimum. A summary that covers all six, with traceable citations for each, is usable. A summary that covers four of them, or covers all six without traceable citations, is not.
The human verification workflow
Traceability makes verification possible. It does not make verification optional. A firm using AI-generated medical summaries needs a documented workflow that every summary passes through before it becomes part of a demand package or filing.
A minimum workflow looks like this:
- Sample verification. The reviewer picks 10–15 sentences at random and clicks through to the source page. Any that fail — either no citation, or a citation that does not match — stops the review. The summary is rejected, not corrected.
- Low-confidence review. Every item the tool flagged as low-confidence is checked manually. This is usually 5–15% of the output.
- Gap verification. Every reported gap in treatment is checked against the original record. False gaps are the most damaging error a summary can contain.
- Pre-existing condition check. Any pre-existing condition the tool flagged is confirmed; and a manual scan of the record for unflagged conditions is done before signing.
- Sign-off. The reviewing paralegal or attorney initials each section they have verified. This is the record that shows the firm supervised the tool's output.
Admissibility and chain of evidence
Medical record summaries produced by AI are generally not themselves evidence. They are attorney work product — internal documents that support the demand or the litigation strategy. That has two consequences:
1. Work product protection helps — if the workflow is clean. A summary that is clearly internal analysis and reviewed by an attorney is more likely to remain protected than one that is indistinguishable from a clinical document. A summary that is presented as if it were a factual medical chronology, without attorney review, is weaker.
2. The underlying records are what matter in court. If the summary is used to prepare a demand or a deposition, and a factual claim in it is challenged, the answer is always the underlying record — the page the claim was extracted from. Which is why traceability is not a nice feature; it is the mechanism by which a summary becomes defensible.
A summary that cannot be traced to a specific page of a specific record is a liability, not an asset. If a claim in it is challenged and you cannot show the source, you have lost more than the point — you have compromised the credibility of everything else in the document.
For the workflow and supervision context, see our guide to how agentic AI is changing plaintiff law firm workflows — the compounding error problem applies to medical records just as it does to case management.
How to evaluate a tool's traceability
During any vendor demonstration, ask the vendor to do the following, live:
| Test | Pass criteria |
|---|---|
| Click a citation | The source page opens, and the cited passage is highlighted |
| Check a hard case | Provide a record set with conflicting dates and see which date the tool picks — and whether it flags the conflict |
| Introduce a gap | Provide a record set with a deliberate treatment gap; the tool should flag it, not skip it |
| Test a hedge | Find a record where the prognosis is hedged ("may require"); the tool should preserve the hedge, not resolve it |
| Remove a page | Delete a page from the record set; the tool should not hallucinate content from the missing page |
| Review an unfamiliar specialty | Provide a record set from a specialty the tool was not trained on specifically; check whether accuracy visibly degrades |
A tool that passes all six tests in a live demonstration is worth serious evaluation. A tool that cannot be tested this way — because the demo environment is locked, or the citations are not clickable, or the vendor deflects — is not ready for litigation use regardless of how polished the product is.
Frequently asked questions
How accurate are AI medical record summaries in 2026?
Vendor-published accuracy figures typically range from 85% to 99% depending on the task and the vendor's own test set, but these figures are not independently verifiable and are not uniform across output types. Treatment dates are usually extracted accurately; prognosis and gap detection are less reliable. The property to evaluate is traceability, not accuracy.
What is traceability in an AI medical record summary?
Traceability means every sentence in the summary links to a specific page and passage in the source record. A summary without traceable citations cannot be safely used in demand packages or litigation, because if a claim is challenged you cannot show where it came from.
Can I use ChatGPT to summarise medical records for a personal injury case?
Not safely. Consumer AI tools do not provide source-linked output, do not have HIPAA-compliant infrastructure without a BAA, and do not meet the confidentiality and supervision standards for handling protected health information. Firms need purpose-built tools that sign a BAA and provide traceable output.
What is the difference between accuracy and traceability?
Accuracy is a statistical property — how often the output is correct against a vendor test set. It cannot be measured on a single case and requires trusting the vendor. Traceability is a structural property — can each claim be traced to its source. It can be verified on any case in minutes without vendor involvement.
Are AI-generated medical summaries discoverable in litigation?
Generally yes, if they are not protected as attorney work product. Firms should assume that summaries and the verification records associated with them may need to be produced. A clean, documented verification workflow strengthens both the work product argument and the defensibility of the underlying analysis.
How much of an AI medical summary needs to be verified by a human?
At minimum, every low-confidence item the tool flags, every reported gap in treatment, every flagged pre-existing condition, and a random sample of 10–15 high-confidence sentences checked against the source. The sample verification is the check that catches systematic errors.
Which vendors offer traceable medical record summaries?
Several tools in the market in 2026 provide source-linked output, including Supio and Filevine's AI features, though the quality and granularity of traceability varies. Firms should evaluate traceability by live demonstration, not by vendor documentation.
What happens if an AI summary contains an error that reaches a filing?
The firm bears the professional responsibility. Under ABA Formal Opinion 512, the duties of competence and supervision extend to AI-assisted work. An error in a filing can result in sanctions, as has already happened in US courts for AI-generated citations.
Related reading on Dedon Tech: how agentic AI is changing plaintiff law firm workflows · HIPAA and SOC 2 compliance for legal AI tools · best AI legal software for US personal injury firms · full site map
0 Comments: