Can expert models for each entity type help each other tag biomedical text?
Letting separately trained disease, chemical and gene taggers pass their predictions to one another raised precision and cut cases where one entity type was mistaken for another.
Source
CollaboNet: collaboration of deep neural networks for biomedical named entity recognition
Study at a glance
- Design
- Computational / modelling — New NER architecture (CollaboNet) compared with single-task and multi-task BiLSTM-CRF baselines on six public biomedical corpora, plus manual error analysis.
- N
- No single N: six benchmark datasets (BC2GM, BC4CHEMD, BC5CDR-chem, BC5CDR-disease, JNLPBA, NCBI); the error analysis covered 4334 single-model errors and 3966 CollaboNet errors.
- Population
- Sentences from MEDLINE abstracts annotated for disease, chemical or gene/protein entities
- Outcome
- Entity-level precision, recall and F1 (exact span match); counts of error types
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors built a single-task tagger (a bidirectional LSTM with a CRF output layer and CNN-based character embeddings) for each entity type. In CollaboNet, each tagger is first trained on its own dataset, then retrained while receiving the outputs of the other 'collaborator' taggers as extra input. They compared this against single-task models and a published multi-task model on six benchmark corpora and hand-classified the errors.
What they found
CollaboNet achieved higher precision and F1 than most single-task models and beat the previous multi-task model on 4 of 6 datasets. In the error analysis, entity-type confusions made up 30.8% of the single model's errors but 18.6% of CollaboNet's errors. However, CollaboNet made more span (boundary) errors, and the authors note the overall gain may look modest given the added complexity.
The limits
What it doesn't show
The extracted text lacks the results tables, so exact F1 scores per dataset cannot be checked here. Improvements are reported against a small set of BiLSTM-era baselines, before transformer models like BioBERT became standard. Training is costly (about 3000 minutes for the collaborative phases), and inconsistent annotation conventions across datasets make exact-match scores partly a measure of labelling style rather than understanding. No significance tests across the ten random initialisations are described in the text.
Key terms
- Named entity recognition (NER)
- Automatically finding and labelling spans of text that name things such as diseases, chemicals or genes.
- BiLSTM-CRF
- A sequence tagger where a bidirectional recurrent network reads the sentence both ways and a conditional random field picks the most consistent sequence of tags.
- Multi-task learning
- Training one model on several related tasks at once so that shared structure helps each task.
- Polysemy
- When one word has several meanings, e.g. 'VHL' naming both a gene and a disease.
- Precision vs recall
- Precision is the share of predicted entities that are correct; recall is the share of true entities that were found.
Flashcards
0 of 10 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What is the main idea behind CollaboNet?
Common questions
How is CollaboNet different from ordinary multi-task learning?
Instead of one shared model trained on everything, it keeps a separate expert per entity type and feeds the other experts' outputs in as extra features, taking turns so each model is the target in turn.
Why do the authors care more about precision than recall?
In large text-mining pipelines, important facts recur, so a missed entity is often caught elsewhere, but a wrong entity type can propagate errors into downstream results like drug-interaction extraction.
Did CollaboNet fix every kind of error?
No. It reduced entity-type confusions but increased span errors, partly because the datasets label entity boundaries inconsistently.
More on Biomedical text mining