Does adding neighbouring-word context improve biomedical NER?
Feeding each word's neighbours into an LSTM tagger modestly beat earlier deep-learning taggers at finding disease and chemical names, but BERT still did better on genes and on cross-corpus tests.
Source
Biomedical named entity recognition using deep neural networks with contextual information
Study at a glance
- Design
- Computational / modelling — New CLSTM-CRF architecture (word-level, character-level and word+char variants with window-size context) trained and tested on NCBI disease, BioCreative II GM and BioCreative V CDR corpora against BiLSTM, BiLSTM-CRF, GRAM-CNN and BERT, using strict, partial and token-level matching.
- N
- No single N: three corpora (NCBI: 793 PubMed abstracts; GM: 20,000 sentences; CDR: 1500 articles); CLSTM robustness checked over 5 training runs.
- Population
- Annotated biomedical abstracts/sentences with disease, gene and chemical mentions.
- Outcome
- Precision, recall and F-score for named entity recognition under strict mention, partial mention and IOB-tag matching; cross-corpus F-score; training time.
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors built CLSTM, a bidirectional LSTM with a conditional random field output layer in which each word (or character) is represented by concatenating its own embedding with those of its neighbours inside a fixed window. They trained word-level, character-level and combined versions on three annotated biomedical corpora covering diseases, genes and chemicals, choosing window sizes on development sets. They compared CLSTM with BiLSTM, BiLSTM-CRF, GRAM-CNN and BERT, repeated training five times to check robustness, analysed errors and tested transfer between two disease corpora.
What they found
On the NCBI disease corpus, the combined CLSTM reached a strict F-score of 85.68%, about 1.5 points above the best previous model (GRAM-CNN at 84.18%), and even its worst of five runs beat GRAM-CNN. On the CDR corpus it reached 86.44%, beating all baselines, but on the gene corpus BERT was slightly better (81.65% vs 81.44%). In cross-corpus tests BERT had higher recall and F-score, while CLSTM had higher precision; most CLSTM errors involved entity boundaries.
The limits
What it doesn't show
Improvements over earlier models are small (often under 2 F-score points, and 0.02 points on one CDR measure), and no significance tests are reported. BERT, a general-domain pre-trained model, matched or beat CLSTM on genes and cross-corpus transfer, so the paper does not show CLSTM is better than pre-trained transformers such as BioBERT, which it does not test. Only three English abstract corpora were used, and different corpora needed different window sizes, which complicates generalisation. CLSTM also needed about 20% more training time than BiLSTM models.
Key terms
- Named entity recognition (NER)
- Automatically finding and labelling names of things (diseases, genes, chemicals) in text.
- BiLSTM
- A recurrent network that reads a sentence both forwards and backwards so each word's representation reflects context on both sides.
- Conditional random field (CRF)
- An output layer that chooses the best sequence of labels jointly, using how likely one tag is to follow another.
- IOB tagging
- Labelling each token as Beginning, Inside or Outside an entity so multi-word names can be recovered.
- Strict vs partial matching
- Strict matching needs the predicted entity's boundaries and type to match exactly; partial matching accepts overlapping boundaries of the right type.
Flashcards
0 of 10 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What architectural idea defines CLSTM?
Common questions
What is new about CLSTM compared with an ordinary BiLSTM-CRF?
Instead of feeding each word or character alone, it concatenates the embeddings of its neighbours within a window, giving the network explicit n-gram-style local context.
Did CLSTM beat BERT?
Not overall. It won on the disease and chemical corpora but BERT was slightly better on gene names and had higher recall and F-score when models were tested on a different corpus from the one they were trained on.
What kinds of mistakes did the model make?
Mostly boundary errors, such as including or dropping adjectives like 'female' or 'sporadic', splitting coordinated phrases like 'pineal and retinal tumours', and handling bracketed acronyms.
More on Biomedical text mining