Skip to content
PaperFren

Biomedical text mining

Does training on many biomedical datasets at once improve NER?

Crichton G, Pyysalo S, Chiu B, et al. · BMC bioinformatics · 2017

Open access · cc by · source: Europe PMC

Letting one neural network learn many biomedical name-tagging datasets together gave modest average gains, larger gains on some datasets, and the biggest benefit when training data was scarce.

Save to libraryView originalAsk PaperFrenStudy view

Study at a glance

Design
Computational / modelling — Controlled comparison of a single-task CNN tagger against two multi-task CNN variants (shared layers with private outputs; POS-tagger-fed dependent model) across 15 NER corpora, plus pairwise, grouped and reduced-training-data experiments.
N
No single N: 15 biomedical NER corpora plus one POS-tagging corpus; pairwise experiment trained 15 x 14 models.
Population
Public biomedical text corpora annotated for anatomy, chemicals, diseases, genes/proteins and species.
Outcome
Exact mention-level macro F1 score on each corpus's test set.

Structured fields used in claim comparison tables when every cited study has a complete layer.

Key findings

Training on all datasets together raised F-scores significantly on five datasets and lowered one for the shared-output model, while the part-of-speech model improved six and hurt none. Grouping datasets that annotate the same entity type produced 16 significant gains out of 27 experiments. The advantage of multi-task learning grew as training data shrank; for two datasets, half the training data plus a helper dataset beat the full data alone. Which dataset helped which was sometimes surprising, e.g. a gene/protein corpus best helped a chemical corpus.

Methodology

The authors built a convolutional neural network that tags each word in a sentence as part of a named entity (such as a gene or disease name) or not. They compared it with two multi-task versions: one sharing all layers except a separate output per dataset, and one feeding in features from a part-of-speech tagger. They trained on 15 public biomedical corpora, tested every pair of datasets, grouped datasets by entity type, and cut the training data to 50%, 25% and 10% to see how multi-task learning behaves with less data.

Limitations

Average improvements were small (under one F-score point overall), and the authors admit it is hard to predict in advance which dataset combinations will help. The models deliberately skip task-specific tricks such as gazetteers, so they are not state-of-the-art systems, and several corpora have no established NER benchmark to compare against. The models cannot handle overlapping annotations of the same term, and the static-embedding CNN setup predates transformer language models.

How this study connects

Role on claims

Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.

  • SupportsBiomedical named entity recognitionconcept

    Training across datasets is a reliable lever with small data.

    Sharing information across datasets or entity types helps, especially when data are scarce: multi-task training gave 16 significant gains in 27 same-entity experiments and helped most as training data shrank; CollaboNet cut entity-type confusions from 30.8% to 18.6% of errors; one LLaMA trained on three datasets beat single-dataset training (0.876 vs 0.762 F1 on NCBI disease).

    Evidence for the claim as stated.

Related papers in this topic

Same topic cluster — not a recommendation engine.