Representation learning
Can a transformer predict which drugs bind which proteins?
Open access · cc by · source: Europe PMC
Breaking drugs and proteins into frequently recurring sub-pieces and modelling how those pieces interact gave better drug-target predictions than earlier deep models, especially when labelled data were scarce.
Study at a glance
- Design
- Computational / modelling — New architecture (subsequence mining + transformer encoders + pairwise interaction map + CNN) compared with baselines across random, unseen-drug/target and scarce-data splits, plus ablations.
- N
- No single N; three datasets (BIOSNAP main, plus DAVIS and BindingDB), each split 7:1:2 into train/validation/test over five random runs.
- Population
- Drug-protein pairs from BIOSNAP, DAVIS and BindingDB
- Outcome
- ROC-AUC and PR-AUC for binary interaction prediction
Structured fields used in claim comparison tables when every cited study has a complete layer.
Key findings
MolTrans had the best ROC-AUC and PR-AUC on all three datasets in the standard split, with up to 25% improvement over the best baseline (PR-AUC on DAVIS). On unseen drugs or targets it was competitive rather than clearly best. When trained on only small fractions of the data it degraded less than DeepDTI and DeepConv-DTI. Ablations showed the transformer, CNN and interaction module each helped, and that mining sub-structures from large unlabelled data beat mining from small datasets.
Methodology
The authors built MolTrans, which first splits drug SMILES strings and protein sequences into recurring sub-structures mined from large unlabelled databases, similar to sub-word tokens in language models. Transformer encoders give each sub-structure a context-aware embedding, a dot-product map scores every drug-piece by protein-piece pair, and a CNN reads that map to output an interaction probability. They compared it with logistic regression, a DNN, GNN-CPI, DeepDTI, DeepDTA and DeepConv-DTI on BIOSNAP, DAVIS and BindingDB, and ran unseen-drug, unseen-target, low-data and ablation experiments.
Limitations
Negative examples were sampled from pairs never recorded as interacting, so some 'negatives' may be real but untested interactions, which can distort metrics. The advantage largely disappears in the harder unseen-drug and unseen-target settings, which are closest to real discovery. Interpretability evidence consists of a few hand-picked interaction maps checked against literature, not a systematic test. Predictions were never validated in a wet lab.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Not yet placed on a claim. This paper has study layers, but no concept page yet cites it as support, challenge, or qualifier.
Related papers in this topic
Same topic cluster — not a recommendation engine.
- Can a small protein language model match much bigger ones?
- Can one language model read protein sequence and structure?
- Can random negative pairs teach a better genome-sorting model?
- Can a self-taught CT model find cancer markers with little data?
- Can a network trained on everyday photos search pathology slides?