Can deep learning judge how good a predicted protein structure is?
A deep belief network that combines existing quality scores ranked predicted protein structures as well as or better than the best single-model methods of its time.
Source
DeepQA: improving the estimation of single protein model quality with deep belief networks
Study at a glance
- Design
- Computational / modelling — Deep belief network (two RBM layers plus logistic output) trained on CASP8-10, 3DRobot and native structures with five-fold cross-validation, blind-tested on CASP11 stage 1 and 2 models and an ab initio decoy set.
- N
- N=84 · 84 CASP11 protein targets for the main blind test; training and testing together used 803 proteins with 216,875 structural models; ab initio validation used 24 targets.
- Population
- Predicted 3D protein structure models from CASP and decoy sets
- Outcome
- Per-target Pearson correlation between predicted and true GDT-TS quality, and per-target loss of the top-ranked model
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors trained DeepQA, a deep belief network built from two layers of restricted Boltzmann machines, to predict how close a single predicted protein structure is to the true one, using 16 input scores describing energy, structure and physico-chemical properties. They compared it with a support vector machine and a shallow neural network trained on the same data, then reduced the inputs to nine features and blind-tested it on CASP11 models and on ab initio decoys from their own modelling tool.
What they found
The deep belief network had the best per-target correlation on both CASP11 stages, beating the SVM and neural network. On stage one its correlation was 0.64, equal to ProQ2, the leading single-model method, with a loss of 0.09; on stage two it had the highest correlation of all single-model methods. Its gains over its own input features were significant for correlation but mostly not for loss.
The limits
What it doesn't show
The improvement over the strongest existing method is small and often a tie, and the gains in top-model loss were mostly not statistically significant. The model relies on hand-crafted scores from other tools rather than learning from raw structure, so it shows little about what deep learning adds beyond feature combination. Results are from one CASP round and the authors' own ab initio decoys, and the paper gives little analysis of why the network works better.
Key terms
- Deep belief network
- A neural network built by stacking restricted Boltzmann machines, each first pre-trained without labels and then the whole network fine-tuned on the supervised task.
- Restricted Boltzmann machine (RBM)
- A two-layer generative network that learns a probability distribution over its inputs; used as a building block for pre-training.
- Single-model quality assessment
- Estimating the accuracy of one predicted protein structure from that model alone, without comparing it to other models in a pool.
- GDT-TS score
- A 0-to-1 measure of how closely a predicted protein structure superimposes on the true experimental structure.
- Per-target loss
- How much worse the model ranked first by a method is than the best model available for that protein.
Flashcards
0 of 9 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What building block forms the hidden layers of DeepQA?
Common questions
Why prefer single-model over consensus quality assessment?
Consensus methods assume the most common structure in a pool is best, so they fail when a pool is dominated by poor models; single-model methods judge each structure on its own.
Is DeepQA clearly better than ProQ2?
Not decisively. It tied ProQ2 on stage one correlation and loss and was best on stage two correlation, so it matches the state of the art rather than leaping past it.
What is pre-training in a deep belief network?
The RBM layers are first trained without labels using contrastive divergence to initialise the weights, and the full network is then fine-tuned with the quality labels.
More on Protein structure prediction