Biomedical text mining
Does adding neighbouring-word context improve biomedical NER?
Open access · cc by · source: Europe PMC
Feeding each word's neighbours into an LSTM tagger modestly beat earlier deep-learning taggers at finding disease and chemical names, but BERT still did better on genes and on cross-corpus tests.
Study at a glance
- Design
- Computational / modelling — New CLSTM-CRF architecture (word-level, character-level and word+char variants with window-size context) trained and tested on NCBI disease, BioCreative II GM and BioCreative V CDR corpora against BiLSTM, BiLSTM-CRF, GRAM-CNN and BERT, using strict, partial and token-level matching.
- N
- No single N: three corpora (NCBI: 793 PubMed abstracts; GM: 20,000 sentences; CDR: 1500 articles); CLSTM robustness checked over 5 training runs.
- Population
- Annotated biomedical abstracts/sentences with disease, gene and chemical mentions.
- Outcome
- Precision, recall and F-score for named entity recognition under strict mention, partial mention and IOB-tag matching; cross-corpus F-score; training time.
Structured fields used in claim comparison tables when every cited study has a complete layer.
Key findings
On the NCBI disease corpus, the combined CLSTM reached a strict F-score of 85.68%, about 1.5 points above the best previous model (GRAM-CNN at 84.18%), and even its worst of five runs beat GRAM-CNN. On the CDR corpus it reached 86.44%, beating all baselines, but on the gene corpus BERT was slightly better (81.65% vs 81.44%). In cross-corpus tests BERT had higher recall and F-score, while CLSTM had higher precision; most CLSTM errors involved entity boundaries.
Methodology
The authors built CLSTM, a bidirectional LSTM with a conditional random field output layer in which each word (or character) is represented by concatenating its own embedding with those of its neighbours inside a fixed window. They trained word-level, character-level and combined versions on three annotated biomedical corpora covering diseases, genes and chemicals, choosing window sizes on development sets. They compared CLSTM with BiLSTM, BiLSTM-CRF, GRAM-CNN and BERT, repeated training five times to check robustness, analysed errors and tested transfer between two disease corpora.
Limitations
Improvements over earlier models are small (often under 2 F-score points, and 0.02 points on one CDR measure), and no significance tests are reported. BERT, a general-domain pre-trained model, matched or beat CLSTM on genes and cross-corpus transfer, so the paper does not show CLSTM is better than pre-trained transformers such as BioBERT, which it does not test. Only three English abstract corpora were used, and different corpora needed different window sizes, which complicates generalisation. CLSTM also needed about 20% more training time than BiLSTM models.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Progress came in small steps on fixed benchmarks.
Each method generation reported gains of a few F-score points on shared corpora: NERBio's additions raised JNLPBA F-score by 3.28 points to 72.98%; GRAM-CNN reached 87.26% on gene mentions; a context-LSTM reached 85.68% on NCBI disease, about 1.5 points above GRAM-CNN.
Evidence for the claim as stated.
Strict F1 partly measures agreement with labelling rules.
Boundary and annotation-convention errors are a large share of what remains: relaxing exact boundary matching raised NERBio's scores by roughly 3–7 points, most CLSTM errors involved boundaries, and GPT's exact-match NER scores were much lower than relaxed ones.
Evidence for the claim as stated.
Whether custom neural architectures still matter once pretrained transformers exist: CLSTM beat GRAM-CNN, but general BERT matched or beat it on genes and cross-corpus transfer, and BioBERT beat plain BERT on every NER dataset.
Evidence for the claim as stated.
Open questions
Tensions this paper is part of
From concept pages' “where studies disagree.” Disagreement means the same question; scope means different assays, populations, or outcomes.
Whether custom neural architectures still matter once pretrained transformers exist: CLSTM beat GRAM-CNN, but general BERT matched or beat it on genes and cross-corpus transfer, and BioBERT beat plain BERT on every NER dataset.
Related papers in this topic
Same topic cluster — not a recommendation engine.
- Can one model find diseases, genes and how they relate in text?
- Does training on many biomedical datasets at once improve NER?
- Can expert models for each entity type help each other tag biomedical text?
- Better tagging of gene and protein names in biology papers
- Does focusing on nearby words help find gene and disease names?