Do deep learned features help predict disease from gene data?
Across dozens of prediction tasks from gene expression data, fancy learned representations did not beat a simple regularised linear model on properly normalised data.
Source
Standard machine learning approaches outperform deep representation learning on phenotype prediction from transcriptomics data
Study at a glance
- Design
- Computational / modelling — Systematic benchmark of normalisations, gene sets, learned representations and classifiers across many RNA-seq prediction tasks using 5-fold nested cross-validation.
- N
- About 37k samples trained the unsupervised representations; supervised tasks (24 classification, 26 survival) each had typically a few hundred samples.
- Population
- Human RNA-seq samples from the recount2 database (GTEx, TCGA and SRA).
- Outcome
- Out-of-sample AUC (binary), accuracy (multiclass) and concordance index (survival).
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
Using public RNA-seq data, the authors set up 24 classification tasks and 26 survival tasks and compared thousands of models with 5-fold nested cross-validation. They varied the normalisation, the gene set, whether features came from PCA, a denoising autoencoder, a variational autoencoder or none, and the classifier (k-nearest neighbours, random forest, regularised logistic regression). They also tested a semi-supervised autoencoder trained jointly with several prediction tasks.
What they found
Models using many genes beat those using the single best gene, and log-ratio normalisation beat raw relative abundances. Representations from PCA, autoencoders and VAEs lowered out-of-sample performance, and the semi-supervised version helped slightly only on tasks similar to its training data. Random forest won 13 tasks and logistic regression nine, but logistic regression was more consistent, so an l2-regularised linear model on centred log-ratio data is recommended as the baseline.
The limits
What it doesn't show
The result covers bulk RNA-seq phenotype prediction with the specific architectures and 512-dimensional embeddings tested; other designs or much larger labelled datasets might change it. Supervised tasks had only a few hundred samples each, which may favour simple models. The authors suggest simple methods may already be near the limit of what bulk expression can predict, but they do not test that directly.
Key terms
- Representation learning
- Training a model, often without labels, to produce new features from raw data that are meant to make later prediction easier.
- Autoencoder
- A neural network trained to compress its input into a small code and reconstruct it; the code is used as learned features.
- Nested cross-validation
- Cross-validation with an inner loop for choosing hyperparameters and an outer loop for measuring performance, avoiding optimistic estimates.
- L2 regularisation
- A penalty on large model weights that reduces overfitting when there are far more features than samples.
- Centred log-ratio transform
- A normalisation for relative (compositional) data that takes logs relative to the sample's average, turning fold-changes into linear differences.
Flashcards
0 of 10 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What was the main finding about learned representations?
Common questions
Why might unsupervised features fail to help prediction?
They capture the biggest sources of variation, but a small, consistent difference that predicts disease can be swamped by larger irrelevant variation and dropped.
Does this mean deep learning is useless for gene data?
No; it means the tested representations gave no consistent benefit for these tasks, so simple regularised models should be the baseline any new method must beat.
Why use nested cross-validation?
Tuning hyperparameters on the same data used to score a model inflates its apparent performance; the nested design keeps the scoring data untouched.
More on Representation learning