AI models lose track of facts buried in the middle of patient records, study finds

Key facts

• A new paper defines the clinical lost-in-the-middle (CLitM) problem and gives what it calls its first systematic characterization, using the MedAlign benchmark across 2,196 instruction-response pairs and six language models.
• Instruction-following accuracy peaks at 59.5% for facts in the 20-30% decile of a patient's EHR timeline and falls to a trough of 37.6% at the 70-80% decile, a 21.9 percentage-point gap; 67.8% of reference answers fall inside that middle trough zone rather than near the record's edges.
• The authors' proposed fix, Query-Conditioned Clinical Suppression (QCCS), is tested against four retrieval baselines, BM25, BM25 with section-header filtering, dense retrieval, and cross-encoder reranking, on 83 held-out instructions with Qwen2.5-7B-Instruct running a 16k context window.
• On middle-position instructions, QCCS reaches 16.7% accuracy versus 3.3% for BM25 and 0.0% for both cross-encoder reranking and dense retrieval; across all instructions, QCCS reaches 25.3% overall versus at most 3.6% for any retrieval-only baseline.
• QCCS's advantage is not explained by retrieval recall: BM25 retrieves the gold evidence sentence in 98.8% of instructions versus 34.9% for QCCS at retrieval depth k=20, yet retrieval-based methods top out at 2.6% accuracy even when they find it, while QCCS reaches 25.0% accuracy even when it does not.

Why it matters

Clinical language-model deployments increasingly have to read an entire patient chart at once, and electronic health records now routinely exceed 100,000 tokens per patient. The paper argues that the well-documented lost-in-the-middle (LitM) effect, where information near the center of a long context is retrieved less reliably than information near the edges, is not a benign quirk in this setting: the single most consequential fact in a clinical note can sit exactly at its center. The authors coin the term clinical lost-in-the-middle (CLitM) for this and say they provide its first systematic characterization, using the MedAlign benchmark across 2,196 instruction-response pairs and six language models.

Who it affects

Anyone building or deploying language models over full electronic health records: clinicians using such tools for chart review or instruction-following tasks, and engineers designing the retrieval or context-selection layer in front of a clinical LLM. The stakes are not theoretical. 67.8% of the benchmark's reference answers fall between the 10th and 90th percentiles of the EHR timeline, meaning most of the real clinical questions the paper studied point to facts sitting inside the exact trough zone where accuracy is lowest, not near the document's edges where models perform best.

How to use it

This is a research proof-of-concept, and the authors frame it that way explicitly: they call it a proof-of-concept evaluation, not something tested on live patient care or reviewed by regulators. The mitigation they test, Query-Conditioned Clinical Suppression (QCCS), is described only as a lightweight query-conditioned selection gate; the text gives no further detail on how it decides what to suppress, and no latency, cost, or compute figures comparing it with the four retrieval baselines it is evaluated against, BM25, BM25 with section-header filtering, dense retrieval, and cross-encoder reranking.

How solid is it

The CLitM characterization draws on 2,196 instruction-response pairs and six language models, with confidence intervals reported for the peak and trough accuracy figures, 59.5% (95% CI 46.3 to 71.0) and 37.6% (95% CI 23.2 to 52.5). The QCCS comparison against the four retrieval baselines uses a separate, smaller set of 83 held-out instructions, scored with an LLM-as-judge, and is reported only for one model, Qwen2.5-7B-Instruct at a 16k context window; the text does not say whether the other five models used in the earlier characterization were also run through this comparison. The authors themselves label the QCCS evaluation a proof-of-concept.

Risks and caveats

The text names no authors, institutional affiliations, or submission date, so the work cannot be attributed beyond the authors here. The QCCS results are demonstrated on a single 7-billion-parameter model at a 16k context window, and the text does not say whether they hold for larger models, longer contexts, or the other five models used earlier in the paper; it also does not define what counts as a middle-position instruction beyond the decile framing used for the CLitM characterization. The result pattern is counterintuitive: BM25 retrieves the gold evidence sentence far more often than QCCS does, 98.8% of instructions versus 34.9% at retrieval depth k=20, yet the retrieval-based methods top out at 2.6% accuracy even when they do retrieve it, while QCCS reaches 25.0% accuracy even when it does not; the authors say this advantage is not explained by retrieval recall, but the text does not detail why it happens. No mechanism, latency, or cost detail is given for QCCS, and the authors call the evaluation a proof-of-concept, not a clinically deployed or regulator-reviewed system.

“In this proof-of-concept evaluation, query-aligned context selection predicts EHR instruction-following accuracy better than gold-sentence retrieval recall.”

— the authors