Does training on many biomedical datasets at once improve NER?
Letting one neural network learn many biomedical name-tagging datasets together gave modest average gains, larger gains on some datasets, and the biggest benefit when training data was scarce.
Source
A neural network multi-task learning approach to biomedical named entity recognition
Study at a glance
- Design
- Computational / modelling — Controlled comparison of a single-task CNN tagger against two multi-task CNN variants (shared layers with private outputs; POS-tagger-fed dependent model) across 15 NER corpora, plus pairwise, grouped and reduced-training-data experiments.
- N
- No single N: 15 biomedical NER corpora plus one POS-tagging corpus; pairwise experiment trained 15 x 14 models.
- Population
- Public biomedical text corpora annotated for anatomy, chemicals, diseases, genes/proteins and species.
- Outcome
- Exact mention-level macro F1 score on each corpus's test set.
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors built a convolutional neural network that tags each word in a sentence as part of a named entity (such as a gene or disease name) or not. They compared it with two multi-task versions: one sharing all layers except a separate output per dataset, and one feeding in features from a part-of-speech tagger. They trained on 15 public biomedical corpora, tested every pair of datasets, grouped datasets by entity type, and cut the training data to 50%, 25% and 10% to see how multi-task learning behaves with less data.
What they found
Training on all datasets together raised F-scores significantly on five datasets and lowered one for the shared-output model, while the part-of-speech model improved six and hurt none. Grouping datasets that annotate the same entity type produced 16 significant gains out of 27 experiments. The advantage of multi-task learning grew as training data shrank; for two datasets, half the training data plus a helper dataset beat the full data alone. Which dataset helped which was sometimes surprising, e.g. a gene/protein corpus best helped a chemical corpus.
The limits
What it doesn't show
Average improvements were small (under one F-score point overall), and the authors admit it is hard to predict in advance which dataset combinations will help. The models deliberately skip task-specific tricks such as gazetteers, so they are not state-of-the-art systems, and several corpora have no established NER benchmark to compare against. The models cannot handle overlapping annotations of the same term, and the static-embedding CNN setup predates transformer language models.
Key terms
- Named entity recognition (NER)
- Finding and labelling mentions of things like genes, chemicals or diseases in text.
- Multi-task learning (MTL)
- Training one model on several related tasks at once so a shared representation can transfer knowledge between them.
- F-score (F1)
- The harmonic mean of precision and recall; here computed on exact entity matches.
- Word embeddings
- Fixed numerical vectors representing words, pre-trained on biomedical text and used as model input.
- Auxiliary task
- A secondary task (here part-of-speech tagging) trained to provide helpful information for the main task.
Flashcards
0 of 11 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What is the core idea of multi-task learning?
Common questions
Why not just merge all the datasets into one big training set?
Different groups annotate differently; earlier work found naive merging dropped performance sharply, so the authors kept each dataset as its own task.
When is multi-task learning most useful here?
When a dataset is small: the gap over single-task training widened as training data was cut.
Which multi-task model is safer?
The dependent (POS-fed) model had smaller average gains but never significantly hurt a dataset, whereas the fully shared model had bigger wins and a loss.
More on Biomedical text mining