Gene expression
Simple baselines beat cell foundation models on Perturb-seq
Open access · cc by · source: Europe PMC
Benchmarking scGPT and scFoundation against baselines finds that training-set mean and basic ML with biological features outperform the foundation models on post-perturbation expression prediction.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Foundation cell models for post-perturbation prediction need careful benchmarks.
Benchmarking foundation cell models for post-perturbation RNA-seq prediction tests how well learned representations generalize after QC and perturbation—downstream of filtering choices.
Evidence for the claim as stated.
A mito-filter methods result, a foundation-model benchmark, and a disease atlas answer different layers of the single-cell stack.
Evidence for the claim as stated.
Open questions
Tensions this paper is part of
From concept pages' “where studies disagree.” Disagreement means the same question; scope means different assays, populations, or outcomes.
A mito-filter methods result, a foundation-model benchmark, and a disease atlas answer different layers of the single-cell stack.
Study at a glance
- Design
- Computational / modelling — Benchmark of scGPT and scFoundation against train-mean and classical ML on public Perturb-seq splits
- N
- Adamson 68,603 cells, Norman 91,205 cells, plus Replogle subsets; 16,239 genes
- Population
- Public CRISPR perturbation scRNA-seq datasets (CRISPRi/a screens)
- Outcome
- Train-mean and feature-based classical ML outperform scGPT/scFoundation on post-perturbation expression prediction under current benchmarks
Structured fields used in claim comparison tables when every cited study has a complete layer.
Inspect source passages
Simple baselines beat cell foundation models on Perturb-seq
These passages occur verbatim in the stored paper text. They support the points listed below; they do not verify every statement in the explanation.
Train-mean baseline outperformed scGPT and scFoundation
Surprisingly, we found that even the simplest baseline model—taking the mean of training examples—outperformed scGPT and scFoundation.
Adamson dataset has 68,603 single cells (CRISPRi)
the Adamson [ 14 ] dataset comprises 68,603 single cells subjected to single perturbation CRISPR interference (CRISPRi).
Key findings
Even taking the mean of training examples beat scGPT and scFoundation. Biologically featured ML models outperformed scGPT by a large margin. Results expose weaknesses in current post-perturbation benchmarks/metrics and foundation-model claims.
Methodology
Csendes et al. fine-tuned/evaluated scGPT and scFoundation against train-mean and classical regressors (e.g., elastic net, kNN, random forest) using GO/feature embeddings on Adamson, Norman, and Replogle Perturb-seq datasets, scoring Pearson correlation in raw and differential expression spaces.
Limitations
Does not prove foundation models useless for all tasks—only that they underperform on these prediction setups. Future architectures/metrics might reverse the ranking.
Related papers in this topic
Same topic cluster — not a recommendation engine.