Topic
Representation learning research, explained
8 open-access representation learning studies, each with a flashcard deck and a quiz.
- Can a small protein language model match much bigger ones?
A compact protein model pretrained to both fill in masked amino acids and predict protein functions performs close to far larger models across many protein tasks.
- Can a transformer predict which drugs bind which proteins?
Breaking drugs and proteins into frequently recurring sub-pieces and modelling how those pieces interact gave better drug-target predictions than earlier deep models, especially when labelled data were scarce.
- Can one language model read protein sequence and structure?
Teaching a protein language model to translate between amino-acid sequences and a letter-code for 3D shape let it find distant structural relatives almost as well as real structures, thousands of times faster than predicting structures first.
- Can random negative pairs teach a better genome-sorting model?
Treating randomly chosen pairs of DNA fragments as 'different' let a contrastive model sort fragments into genomes better and far more cheaply than using reference-database labels.
- Can a self-taught CT model find cancer markers with little data?
A CT image model pretrained without labels produced features that, with just a simple linear classifier, matched or beat fully supervised deep networks and held up best when training data were cut to a tenth.
- Can a network trained on everyday photos search pathology slides?
A neural network that had never seen tissue images could still find visually similar pathology patches far better than chance and better than a classic hand-crafted method.
- Can a pre-trained medical AI cope with a new disease using few labels?
A model pre-trained on large amounts of unlabelled medical images and text could perform COVID-19 reporting, diagnosis and prognosis with only about 1% of the usual labelled data.
- Do deep learned features help predict disease from gene data?
Across dozens of prediction tasks from gene expression data, fancy learned representations did not beat a simple regularised linear model on properly normalised data.