Simple baselines beat cell foundation models on Perturb-seq
Benchmarking scGPT and scFoundation against baselines finds that training-set mean and basic ML with biological features outperform the foundation models on post-perturbation expression prediction.
Source
Benchmarking foundation cell models for post-perturbation RNA-seq prediction
What they did
Csendes et al. fine-tuned/evaluated scGPT and scFoundation against train-mean and classical regressors (e.g., elastic net, kNN, random forest) using GO/feature embeddings on Adamson, Norman, and Replogle Perturb-seq datasets, scoring Pearson correlation in raw and differential expression spaces.
What they found
Even taking the mean of training examples beat scGPT and scFoundation. Biologically featured ML models outperformed scGPT by a large margin. Results expose weaknesses in current post-perturbation benchmarks/metrics and foundation-model claims.
The limits
What it doesn't show
Does not prove foundation models useless for all tasks—only that they underperform on these prediction setups. Future architectures/metrics might reverse the ranking.
Key terms
- Foundation cell model
- Large pretrained scRNA-seq model fine-tuned for downstream tasks.
- scGPT / scFoundation
- Transformer-style foundation models benchmarked here.
- Post-perturbation prediction
- Predicting expression after a known genetic/chemical perturbation.
- Train-mean baseline
- Predicting the average of training examples—surprisingly strong here.
- Perturb-seq
- Single-cell RNA-seq readouts of CRISPR perturbations.
- Differential expression space
- Perturbed profile minus control, used for correlation metrics.
Flashcards
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
Which baseline beat scGPT/scFoundation?
Common questions
Which foundation models were tested?
scGPT and scFoundation versus simple baselines.
What simple baseline won?
Taking the mean of training examples outperformed both foundation models.
Did classical ML help?
Yes—models with biologically meaningful features beat scGPT by a large margin.
Example dataset sizes?
Adamson ~68.6k cells; Norman ~91.2k cells.
More on Gene expression