Simple baselines beat cell foundation models on Perturb-seq
Benchmarking scGPT and scFoundation against baselines finds that training-set mean and basic ML with biological features outperform the foundation models on post-perturbation expression prediction.
Source
Benchmarking foundation cell models for post-perturbation RNA-seq prediction
Study at a glance
- Design
- Computational / modelling — Benchmark of scGPT and scFoundation against train-mean and classical ML on public Perturb-seq splits
- N
- Adamson 68,603 cells, Norman 91,205 cells, plus Replogle subsets; 16,239 genes
- Population
- Public CRISPR perturbation scRNA-seq datasets (CRISPRi/a screens)
- Outcome
- Train-mean and feature-based classical ML outperform scGPT/scFoundation on post-perturbation expression prediction under current benchmarks
Structured fields used in claim comparison tables when every cited study has a complete layer.
Inspect source passages
Simple baselines beat cell foundation models on Perturb-seq
These passages occur verbatim in the stored paper text. They support the points listed below; they do not verify every statement in the explanation.
Train-mean baseline outperformed scGPT and scFoundation
Surprisingly, we found that even the simplest baseline model—taking the mean of training examples—outperformed scGPT and scFoundation.
Adamson dataset has 68,603 single cells (CRISPRi)
the Adamson [ 14 ] dataset comprises 68,603 single cells subjected to single perturbation CRISPR interference (CRISPRi).
What they did
Csendes et al. fine-tuned/evaluated scGPT and scFoundation against train-mean and classical regressors (e.g., elastic net, kNN, random forest) using GO/feature embeddings on Adamson, Norman, and Replogle Perturb-seq datasets, scoring Pearson correlation in raw and differential expression spaces.
What they found
Even taking the mean of training examples beat scGPT and scFoundation. Biologically featured ML models outperformed scGPT by a large margin. Results expose weaknesses in current post-perturbation benchmarks/metrics and foundation-model claims.
The limits
What it doesn't show
Does not prove foundation models useless for all tasks—only that they underperform on these prediction setups. Future architectures/metrics might reverse the ranking.
Key terms
- Foundation cell model
- Large pretrained scRNA-seq model fine-tuned for downstream tasks.
- scGPT / scFoundation
- Transformer-style foundation models benchmarked here.
- Post-perturbation prediction
- Predicting expression after a known genetic/chemical perturbation.
- Train-mean baseline
- Predicting the average of training examples—surprisingly strong here.
- Perturb-seq
- Single-cell RNA-seq readouts of CRISPR perturbations.
- Differential expression space
- Perturbed profile minus control, used for correlation metrics.
Flashcards
0 of 8 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
Which baseline beat scGPT/scFoundation?
Common questions
Which foundation models were tested?
scGPT and scFoundation versus simple baselines.
What simple baseline won?
Taking the mean of training examples outperformed both foundation models.
Did classical ML help?
Yes—models with biologically meaningful features beat scGPT by a large margin.
Example dataset sizes?
Adamson ~68.6k cells; Norman ~91.2k cells.
More on Gene expression