Concept
Biomedical named entity recognition
9 studies1 discoveryEvidence last moved Sep 27, 2026
Biomedical named entity recognition finds and labels mentions of things like genes, proteins, diseases and chemicals in text, usually as the first step before relation extraction. The papers here are benchmark studies spanning hand-engineered CRF systems (2006–2008), CNN/LSTM neural taggers (2017–2019), pretrained transformers and instruction-tuned LLMs.
It shows how a single task evolved across three generations of methods, and teaches why F-scores on these corpora must be read carefully: many 'errors' are disagreements about boundaries and annotation conventions.
Studies
9
Findings
4
10 supporting · 0 challenging · 0 qualifying citations
Open tensions
1
Latest change
Concept page published
Biomedical named entity recognition
Currently
What we know
- Progress came in small steps on fixed benchmarks.
- Training across datasets is a reliable lever with small data.
- Strict F1 partly measures agreement with labelling rules.
- Before pretraining, dictionaries carried much of the knowledge.
Largest unresolved question
Whether custom neural architectures still matter once pretrained transformers exist: CLSTM beat GRAM-CNN, but general BERT matched or beat it on genes and cross-corpus transfer, and BioBERT beat plain BERT on every NER dataset.
Common misconceptions
A one-point F-score gain means a clearly better system.
Many gains were under two points with no significance testing (NERBio, CLSTM, BioBERT), and corpora contain inconsistent annotations that cap achievable scores.
Adding more datasets to training always helps.
Multi-task training raised some datasets but lowered one, and the authors found it hard to predict which combinations would help.
Related
Claim ledger
What the evidence shows
Drawn from 9 studies in this library. Mix labels say which citation roles are present; they are not a strength score. Supports means evidence for a finding; Challenges means evidence against a stated position; Qualifies marks scope.
Progress came in small steps on fixed benchmarks.
Each method generation reported gains of a few F-score points on shared corpora: NERBio's additions raised JNLPBA F-score by 3.28 points to 72.98%; GRAM-CNN reached 87.26% on gene mentions; a context-LSTM reached 85.68% on NCBI disease, about 1.5 points above GRAM-CNN.
- Better tagging of gene and protein names in biology papers
- Does focusing on nearby words help find gene and disease names?
- Does adding neighbouring-word context improve biomedical NER?
Study Role Design N Population Outcome Better tagging of gene and protein names in biology papers Supports Computational / modellingIncremental feature study with a linear-chain CRF on the JNLPBA 2004 (GENIA-derived) benchmark, compared with the top shared-task systems. N=404 · 404 newly annotated MEDLINE test abstracts; training used the GENIA-derived JNLPBA training set of 2,000 abstracts. MEDLINE abstracts on human blood cells and transcription factors Exact-match F-score for five entity types (protein, DNA, RNA, cell line, cell type) Does focusing on nearby words help find gene and disease names? Supports Computational / modellingNew neural architecture (word/character/POS embeddings, multi-size CNN kernels, CRF output) compared with published NER systems on three benchmark corpora. No single N: BC2 has 15 000 training and 5000 test sentences; NCBI has 6892 disease mentions from 793 abstracts; JNLPBA has 20 546 training and 4260 test sentences. Biomedical text corpora (gene mentions, disease mentions, and five entity classes such as protein, DNA and cell type) Precision, recall and F1-score for entity recognition Does adding neighbouring-word context improve biomedical NER? Supports Computational / modellingNew CLSTM-CRF architecture (word-level, character-level and word+char variants with window-size context) trained and tested on NCBI disease, BioCreative II GM and BioCreative V CDR corpora against BiLSTM, BiLSTM-CRF, GRAM-CNN and BERT, using strict, partial and token-level matching. No single N: three corpora (NCBI: 793 PubMed abstracts; GM: 20,000 sentences; CDR: 1500 articles); CLSTM robustness checked over 5 training runs. Annotated biomedical abstracts/sentences with disease, gene and chemical mentions. Precision, recall and F-score for named entity recognition under strict mention, partial mention and IOB-tag matching; cross-corpus F-score; training time. Training across datasets is a reliable lever with small data.
Sharing information across datasets or entity types helps, especially when data are scarce: multi-task training gave 16 significant gains in 27 same-entity experiments and helped most as training data shrank; CollaboNet cut entity-type confusions from 30.8% to 18.6% of errors; one LLaMA trained on three datasets beat single-dataset training (0.876 vs 0.762 F1 on NCBI disease).
- Does training on many biomedical datasets at once improve NER?
- Can expert models for each entity type help each other tag biomedical text?
- Can a general chatbot-style model be taught to spot biomedical terms?
Study Role Design N Population Outcome Does training on many biomedical datasets at once improve NER? Supports Computational / modellingControlled comparison of a single-task CNN tagger against two multi-task CNN variants (shared layers with private outputs; POS-tagger-fed dependent model) across 15 NER corpora, plus pairwise, grouped and reduced-training-data experiments. No single N: 15 biomedical NER corpora plus one POS-tagging corpus; pairwise experiment trained 15 x 14 models. Public biomedical text corpora annotated for anatomy, chemicals, diseases, genes/proteins and species. Exact mention-level macro F1 score on each corpus's test set. Can expert models for each entity type help each other tag biomedical text? Supports Computational / modellingNew NER architecture (CollaboNet) compared with single-task and multi-task BiLSTM-CRF baselines on six public biomedical corpora, plus manual error analysis. No single N: six benchmark datasets (BC2GM, BC4CHEMD, BC5CDR-chem, BC5CDR-disease, JNLPBA, NCBI); the error analysis covered 4334 single-model errors and 3966 CollaboNet errors. Sentences from MEDLINE abstracts annotated for disease, chemical or gene/protein entities Entity-level precision, recall and F1 (exact span match); counts of error types Can a general chatbot-style model be taught to spot biomedical terms? Supports Computational / modellingInstruction tuning of LLaMA-1/LLaMA-2 7B on three NER training sets turned into instruction prompts; evaluated on held-out test sets and three unseen datasets against baselines. N=22484 · Instruction-following demonstrations used for fine-tuning, built from the training splits of NCBI disease, BC5CDR-Chemical and BC2GM; test sets and three further generalisation datasets are separate. Sentences from biomedical abstracts and articles annotated for disease, chemical and gene mentions Strict (exact-match) and partial-match precision, recall and F1 for named entity recognition Strict F1 partly measures agreement with labelling rules.
Boundary and annotation-convention errors are a large share of what remains: relaxing exact boundary matching raised NERBio's scores by roughly 3–7 points, most CLSTM errors involved boundaries, and GPT's exact-match NER scores were much lower than relaxed ones.
- Better tagging of gene and protein names in biology papers
- Does adding neighbouring-word context improve biomedical NER?
- Can better prompts make GPT good at spotting medical terms?
Study Role Design N Population Outcome Better tagging of gene and protein names in biology papers Supports Computational / modellingIncremental feature study with a linear-chain CRF on the JNLPBA 2004 (GENIA-derived) benchmark, compared with the top shared-task systems. N=404 · 404 newly annotated MEDLINE test abstracts; training used the GENIA-derived JNLPBA training set of 2,000 abstracts. MEDLINE abstracts on human blood cells and transcription factors Exact-match F-score for five entity types (protein, DNA, RNA, cell line, cell type) Does adding neighbouring-word context improve biomedical NER? Supports Computational / modellingNew CLSTM-CRF architecture (word-level, character-level and word+char variants with window-size context) trained and tested on NCBI disease, BioCreative II GM and BioCreative V CDR corpora against BiLSTM, BiLSTM-CRF, GRAM-CNN and BERT, using strict, partial and token-level matching. No single N: three corpora (NCBI: 793 PubMed abstracts; GM: 20,000 sentences; CDR: 1500 articles); CLSTM robustness checked over 5 training runs. Annotated biomedical abstracts/sentences with disease, gene and chemical mentions. Precision, recall and F-score for named entity recognition under strict mention, partial mention and IOB-tag matching; cross-corpus F-score; training time. Can better prompts make GPT good at spotting medical terms? Supports Computational / modellingIncremental prompt ablation for GPT-3.5 and GPT-4 at temperature 0 (zero-, 1- and 5-shot), compared with fine-tuned BioClinicalBERT and a CRF on held-out test sets. Two corpora: 163 synthetic discharge summaries (MTSamples) and 91 vaccine adverse-event reports (VAERS), each split into train, validation and test. Synthetic clinical notes and public VAERS safety reports Precision, recall and F1 for clinical entity extraction under exact and relaxed matching Before pretraining, dictionaries carried much of the knowledge.
Domain knowledge features mattered a lot in pre-neural systems: in a CRF relation extractor, adding a dictionary window feature lifted F-measure from 39.65 to 67.38.
Debates
Tensions and limits
Some items are genuine disagreements on the same question. Others mark different assays, populations, or outcomes.
Whether custom neural architectures still matter once pretrained transformers exist: CLSTM beat GRAM-CNN, but general BERT matched or beat it on genes and cross-corpus transfer, and BioBERT beat plain BERT on every NER dataset.
Qualified studies asking the same question reach different answers. The disagreement is listed, not scored.
Timeline
How understanding moved
Study years are when the paper was published. Evidence edits are dated changes to this page's claims. Explanations are when PaperFren added a Discovery — not a claim that the science happened that day.
2026
- Prompted GPT models approach fine-tuned clinical models on loose matches but slip on exact boundaries and rules
Concept page published
Biomedical named entity recognition
Change log
What changed
Dated edits to this page's evidence: studies added or removed from a claim, claims added or withdrawn, and new explanations tagged here. Rewordings are not listed.
- Concept page published
Papers
9 studies in this library bear on Biomedical named entity recognition, ordered by citations.
- Does pre-training BERT on biomedical papers help it read biology?
Letting a general language model keep learning from millions of biomedical abstracts and articles made it clearly better at biomedical text tasks than the same model trained only on Wikipedia and books.
- Can better prompts make GPT good at spotting medical terms?
Carefully built prompts made GPT-4 nearly as good as a fine-tuned clinical model at loosely matching medical terms, but it still fell clearly short on exact boundaries.
- Can one model find diseases, genes and how they relate in text?
Treating relation extraction as a sequence-labelling problem with conditional random fields matched or beat older methods, and relation-specific features drove most of the gain.
- Does training on many biomedical datasets at once improve NER?
Letting one neural network learn many biomedical name-tagging datasets together gave modest average gains, larger gains on some datasets, and the biggest benefit when training data was scarce.
- Can expert models for each entity type help each other tag biomedical text?
Letting separately trained disease, chemical and gene taggers pass their predictions to one another raised precision and cut cases where one entity type was mistaken for another.
- Can a general chatbot-style model be taught to spot biomedical terms?
A general-purpose open language model, fine-tuned on instructions made from existing labelled data, found diseases, chemicals and genes in biomedical text about as well as a specialist biomedical model and far better than prompted GPT models.
- Better tagging of gene and protein names in biology papers
Adding chosen word-pair features, replacing numbers with placeholders and applying sentence-wide patterns lifted a CRF biomedical name tagger above every system in a standard shared task.
- Does adding neighbouring-word context improve biomedical NER?
Feeding each word's neighbours into an LSTM tagger modestly beat earlier deep-learning taggers at finding disease and chemical names, but BERT still did better on genes and on cross-corpus tests.
Show 1 more studiesShow fewer studies
- Does focusing on nearby words help find gene and disease names?
A convolutional network that looks only at the words around each term matched or beat most existing tools at spotting biomedical names in text, without hand-made features.
Compare studies
Select 2–10 studies. Design and N are labels, not a ranking.
Nothing selected yet.
Questions
What is still open
Whether custom neural architectures still matter once pretrained transformers exist: CLSTM beat GRAM-CNN, but general BERT matched or beat it on genes and cross-corpus transfer, and BioBERT beat plain BERT on every NER dataset.
Ask PaperFren about Biomedical named entity recognition
Study this conceptflashcards and short-answer questions
Explain why exact-match F1 can understate a biomedical NER system's usefulness.
Exact matching counts a mention wrong if its boundaries differ even slightly. NERBio scores rose about 3–7 points when boundaries were relaxed, and GPT models scored far lower on exact than relaxed matching. Several papers also document inconsistent annotation in the corpora themselves, so part of the error reflects labelling conventions.
When is multi-task learning most useful for BioNER?
The 2017 multi-task study found the advantage grew as training data shrank, and half the data plus a helper dataset could beat the full data alone. Instruction-tuned LLaMA similarly gained from combined training. Average gains were small, though, and not every combination helped.
Flashcards
0 of 3 answers reviewed