Neural network models of the brain
Could brain-like local learning rules approximate backpropagation?
Open access · cc by · source: Europe PMC
A predictive coding network that changes each synapse using only the activity of the two neurons it connects can learn almost exactly like backpropagation, reaching the same accuracy on handwritten digits.
Study at a glance
- Design
- Computational / modelling — Mathematical analysis plus simulations comparing weight updates and learning curves of a predictive coding network and a standard backprop network (toy 1-1-1 net, 784-600-600-10 MNIST net, bidirectional linear association task).
- N
- No single N; MNIST learning curves are averaged over 10 simulations per model, and toy tasks use randomly generated samples.
- Population
- Simulated artificial neural networks trained on synthetic data and MNIST handwritten digits
- Outcome
- Angle between predictive coding and backprop weight updates; MNIST training and validation error; prediction error in both input-to-output and output-to-input directions
Structured fields used in claim comparison tables when every cited study has a complete layer.
Key findings
In prediction mode the network settles to exactly the same activities as the equivalent artificial network. Its weight updates point in nearly the same direction as backprop near a good solution, and converge to backprop as the variance parameter on the output layer is made large. On MNIST all networks reached 0.00% training error and 1.7% to 1.8% validation error, with learning curves similar to backprop. With equal noise assumed on inputs and outputs, the network predicted reasonably well in both directions, whereas backprop is optimal only for input-to-output prediction.
Methodology
The authors built a supervised predictive coding network in which separate error nodes signal the mismatch between each layer and the prediction from the layer above, and weights change by a simple Hebbian product of pre- and postsynaptic activity. They proved when its updates match backpropagation and checked this in a tiny three-layer network, then trained a 784-600-600-10 version on MNIST and compared it with an ordinary backprop network. They also tested an architecture where inputs and outputs both enter at the bottom, letting the network predict in both directions.
Limitations
Exact equivalence with backprop happens only in a limit where error-node activity becomes tiny, which the authors admit may be hard for spiking neurons to signal reliably. The model assumes symmetric forward and backward weights, one-to-one pairing of value and error neurons, and error neurons that can go negative, none of which is established in cortex. MNIST learning rates were not tuned and only small fully connected networks were tested, so it says nothing about scaling to modern deep architectures. It is a simulation study with no neural recordings, so it shows biological feasibility, not that the brain actually learns this way.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Backprop-like learning is possible with local rules, in principle.
Brain-like local learning rules can approximate backpropagation: a predictive-coding network with Hebbian plasticity converged to backprop's updates in a limit and reached 1.7–1.8% MNIST validation error.
Evidence for the claim as stated.
Related papers in this topic
Same topic cluster — not a recommendation engine.