Skip to content
PaperFren

Clinical NLP

Can smarter retrieval help LLMs read long clinical notes?

Lopez I, Swaminathan A, Vedula K, et al. · NPJ digital medicine · 2025

Open access · cc by · source: Europe PMC

Retrieving only the passages around clinically relevant terms let language models extract patient information slightly more accurately while using far less text and time than feeding whole notes or embedding-based chunks.

Save to libraryView originalAsk PaperFrenStudy view

Study at a glance

Design
Computational / modelling — Pipeline of zero-shot NER (Flan-T5), embedding + GPT-4 entity filtering, UMLS/GPT-4 synonym augmentation and 150-word context windows, fed to six LLMs for 5-shot variable extraction; compared with BGE chunk-embedding RAG (top-3/5/10) and full-note chunking, plus weak-labelling and BERT-distillation baselines.
N
No single N: Stanford MOUD cohort (767 training and 505 testing patients) with 420 physician-annotated test notes for 13 variables; CheXpert subset with 1000 annotated test radiology reports for 5 findings; NER sensitivity checked on 1382 NCBI and 450 MOUD entities.
Population
Electronic health record notes from Stanford patients treated for opioid use disorder, and Stanford chest X-ray radiology reports.
Outcome
F1 (and sensitivity, specificity, PPV, NPV) for extracting presence/absence/uncertainty of clinical variables; inference time, model calls and input tokens per note; AUC of distilled BERT classifiers.

Structured fields used in claim comparison tables when every cited study has a complete layer.

Key findings

Averaged across models, CLEAR reached an F1 of 0.90 on the opioid-cohort variables and 0.96 on the radiology findings, and beat chunk-embedding and full-note approaches for all models (e.g. GPT-4: 0.97 vs 0.88 vs 0.90). It needed 1.68 model calls per note versus 4.94 for chunk embedding, used 71% fewer input tokens than chunk embedding and 81% fewer than full notes, and ran faster. Removing the initial NER step dropped F1 from 0.97 to 0.86, and BERT models fine-tuned on CLEAR labels performed within the range of the large models.

Methodology

The authors built CLEAR, a retrieval pipeline that first finds all clinical terms in a note with a zero-shot language model, keeps those related to the question (for example 'depression'), adds synonyms from a medical ontology and GPT-4, and then passes only the 150 words around each match to a language model. They compared it with a standard embedding-based retrieval method and with feeding the full note, using six LLMs to extract 13 variables from opioid-use-disorder clinic notes and 5 findings from chest X-ray reports, judged against physician labels. They also used CLEAR's outputs to train a small BERT classifier.

Limitations

The accuracy advantage over alternatives is small for several models (e.g. Flan-T5: 0.91 vs 0.88), and with ten retrieved chunks the embedding method nearly caught up. All data come from one hospital, the opioid test set was split by date so it contains more COVID-era notes, and only information-extraction tasks were tested, not summarisation or reasoning. Chunk sizes and prompts were not fully tuned, so a better-configured embedding baseline might close the gap. The pipeline also depends on GPT-4 for entity filtering and synonym generation.

How this study connects

Role on claims

Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.

  • SupportsClinical information extraction from notesconcept

    Good retrieval is part of the extraction system, not an afterthought.

    How text is fed to the model matters: retrieving entity-centred 150-word windows (CLEAR) beat chunk-embedding and full-note input for all six LLMs tested while using 71–81% fewer tokens, and removing the entity-recognition step dropped GPT-4 F1 from 0.97 to 0.86.

    Evidence for the claim as stated.

  • QualifiesClinical information extraction from notesconcept

    Good retrieval is part of the extraction system, not an afterthought.

    How text is fed to the model matters: retrieving entity-centred 150-word windows (CLEAR) beat chunk-embedding and full-note input for all six LLMs tested while using 71–81% fewer tokens, and removing the entity-recognition step dropped GPT-4 F1 from 0.97 to 0.86.

    Scope note — Single hospital; the margin over a ten-chunk embedding baseline was small for some models.

    Limits the claim's scope: a different population, assay, or outcome.

Related papers in this topic

Same topic cluster — not a recommendation engine.