Can a transformer predict which cancer drug pairs work together?
Describing drugs by the proteins they affect and cells by which genes they depend on let a transformer predict drug-pair synergy better than an earlier chemistry-based network, while keeping predictions explainable gene by gene.
Source
TranSynergy: Mechanism-driven interpretable deep neural network for the synergistic prediction and pathway deconvolution of drug combinations
Study at a glance
- Design
- Computational / modelling — Transformer-based synergy predictor benchmarked against DeepSynergy using nested cross-validation with leave-combination-out, leave-cell-out and leave-drug-out splits, plus feature ablations.
- N
- N=18553 · 18553 drug-pair-by-cell-line synergy measurements (36 drugs, 35 cancer cell lines) from the Merck screening dataset.
- Population
- Merck high-throughput anti-cancer drug combination screen (36 drugs, 35 cell lines).
- Outcome
- Error and correlation between predicted and measured synergy scores (MSE, Pearson, Spearman) and classification PR-AUC/ROC-AUC.
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors built TranSynergy, a model that represents each drug as a target profile spread across a protein interaction network (via random walk with restart) and each cancer cell line as gene-dependency and/or gene-expression values over 2401 genes. A self-attention transformer, without positional encoding, models interactions between genes before a fully connected layer outputs a synergy score. They compared it with DeepSynergy on 18553 measurements under three splits of increasing difficulty and ran ablations on the input representations. They then used SHAP values ranked over genes to run a gene set enrichment analysis (SA-GSEA) and scored 3650 untested drug pairs.
What they found
In the leave-combination-out setting TranSynergy reached an MSE of 232, significantly lower than DeepSynergy, with correlations improved by 3% (Pearson) and 5% (Spearman) and PR-AUC improved by 5.6%, though ROC-AUC improved only slightly. It beat the baseline in most tissues but not prostate, whose only cell line was unusual. Network-propagated target profiles beat raw binary targets, gene dependency beat expression alone, and adding chemical-structure features did not help. When whole cell lines or drugs were held out, both models dropped sharply and TranSynergy's edge was only slight for new cell lines.
The limits
What it doesn't show
All results come from a single screening dataset with only 36 drugs and 35 cell lines, so generalisation to other screens, synergy scoring methods or patient-derived cells is untested. The model needs known drug targets, so it cannot handle new compounds whose targets are unknown without an extra prediction step. The SHAP-based pathway explanations and the predicted novel pairs are hypotheses; no wet-lab experiments were run to confirm them. Performance in the realistic cold-start settings was much lower than in the headline evaluation.
Key terms
- Drug synergy
- When two drugs together kill cancer cells more than expected from their separate effects; measured here as a continuous synergy score.
- Random walk with restart
- A network propagation algorithm that spreads a drug's known protein targets to nearby proteins in an interaction network, estimating indirect effects.
- Self-attention
- A transformer operation where every input element (here, every gene) is weighted by its relevance to every other, letting the model learn interactions.
- Cold-start evaluation
- Testing on cell lines or drugs never seen in training, which is harder and more realistic than holding out only drug pairs.
- SHAP value
- A game-theory-based score of how much each input feature pushed a particular prediction up or down.
Flashcards
0 of 10 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
Which input representation of cell lines worked best?
Common questions
Why remove the positional encoding from the transformer?
In text the order of words matters, but here each row is a gene and the order of genes is arbitrary, so position information would add meaningless signal.
Does this model discover new cancer treatments?
Not directly. It ranks untested drug pairs and suggests pathways, but these are candidates that still need laboratory testing.
Why did performance drop for held-out cell lines?
Test cell lines were chosen to be as different as possible from training ones, so the model had to extrapolate; the authors suggest pretraining might help.
More on Interpretability