Concept · artificial-intelligence
Deep networks as models of the brain
4 studies1 discoveryEvidence last moved Sep 27, 2026
This concept covers using artificial neural networks to explain how real neurons represent information, and asking whether brain-like circuits could learn the way deep networks do. The evidence is computational: CNNs fitted to macaque V1 and human/monkey IT recordings, and simulations of predictive-coding and successor-representation learning.
Claims that 'CNNs work like the visual cortex' are everywhere. These studies show what prediction accuracy does and does not imply, and why a network explaining data is not the same as the brain using that algorithm.
Studies
4
Findings
4
4 supporting · 0 challenging · 0 qualifying citations
Open tensions
1
Latest change
Concept page published
Deep networks as models of the brain
Currently
What we know
- CNN features beat hand-designed filters for V1, but leave half unexplained.
- Category-trained networks capture IT's structure; unsupervised ones did not.
- Backprop-like learning is possible with local rules, in principle.
- Matching behaviour is a hypothesis about the brain, not a measurement of it.
Largest unresolved question
Supervision is essential for IT but not obviously for V1: the IT study found only the supervised network matched, while in V1 a network trained only on neural data (no object labels) came close to VGG (49.8% vs 51.6%) though it needed more data.
Common misconceptions
If a CNN predicts V1, then V1 was trained for object recognition.
The V1 authors state better prediction shows similar representation, not that V1 is trained on object recognition.
Predictive coding proves the brain does backprop.
The model needs symmetric weights and paired error neurons not established in cortex; it shows feasibility in simulation, not what the brain does.
Related
Claim ledger
What the evidence shows
Drawn from 4 studies in this library. Mix labels say which citation roles are present; they are not a strength score. Supports means evidence for a finding; Challenges means evidence against a stated position; Qualifies marks scope.
CNN features beat hand-designed filters for V1, but leave half unexplained.
Deep CNN features predict early visual cortex better than classic models: a VGG layer explained 51.6% of explainable variance in macaque V1, a data-driven CNN 49.8%, a Gabor filter bank 45.6% and an LNP model 16.3%.
Category-trained networks capture IT's structure; unsupervised ones did not.
Supervised training on object categories matters for higher visual cortex: unsupervised models correlated weakly with IT and none showed IT's animate/inanimate split, while the deep supervised network matched human IT better, and a reweighted version reached the noise ceiling.
Backprop-like learning is possible with local rules, in principle.
Brain-like local learning rules can approximate backpropagation: a predictive-coding network with Hebbian plasticity converged to backprop's updates in a limit and reached 1.7–1.8% MNIST validation error.
Matching behaviour is a hypothesis about the brain, not a measurement of it.
Algorithms can be matched to behavioural signatures without neural data: successor-representation agents reproduce which revaluation tasks animals pass or fail, and the proposed striatal mapping is presented as a hypothesis.
Debates
Tensions and limits
Some items are genuine disagreements on the same question. Others mark different assays, populations, or outcomes.
Supervision is essential for IT but not obviously for V1: the IT study found only the supervised network matched, while in V1 a network trained only on neural data (no object labels) came close to VGG (49.8% vs 51.6%) though it needed more data.
Supervision is essential for IT but not obviously for V1: the IT study found only the supervised network matched, while in V1 a network trained only on neural data (no object labels) came close to VGG (49.8% vs 51.6%) though it needed more data.
- Which vision models represent objects like the brain's IT cortex?
- Do deep neural networks predict early visual neurons best?
Study Role Design N Population Outcome Which vision models represent objects like the brain's IT cortex? Supports Computational / modellingRepresentational similarity analysis comparing model RDMs with existing human fMRI and monkey cell-recording RDMs for 96 object images, plus cross-validated feature reweighting 37 model representations were compared on a set of 96 images. The brain data were reused from earlier human fMRI and monkey recording studies, and the monkey data came from two animals. Computational vision models, compared with human IT (fMRI) and monkey IT (cell recordings) Kendall tau-a correlation between model and IT representational dissimilarity matrices, and a categoricality index Do deep neural networks predict early visual neurons best? Supports Computational / modellingModel comparison on held-out images: pretrained VGG-19 features with a regularised GLM readout, CNNs fitted directly to spikes, and LNP and Gabor filter bank baselines N=166 · 166 V1 neurons, selected from 262 isolated in 17 sessions, went into the models. The neurons came from two monkeys. V1 neurons in two awake, fixating rhesus macaques viewing natural images and synthesised textures Fraction of explainable variance explained (FEV) in spike counts for held-out images
PaperFren reads this as a limit on how far one study travels — different assays, populations, or outcomes — not a forced fight between papers.
Timeline
How understanding moved
Study years are when the paper was published. Evidence edits are dated changes to this page's claims. Explanations are when PaperFren added a Discovery — not a claim that the science happened that day.
2026
- Deep networks trained on object labels predict visual cortex better than hand-built and unsupervised models
Concept page published
Deep networks as models of the brain
Change log
What changed
Dated edits to this page's evidence: studies added or removed from a claim, claims added or withdrawn, and new explanations tagged here. Rewordings are not listed.
- Concept page published
Papers
4 studies in this library bear on Deep networks as models of the brain, ordered by citations.
- Which vision models represent objects like the brain's IT cortex?
Only a deep network trained with a million labelled images came close to the category structure of inferior temporal cortex; unsupervised and hand-engineered models did not.
- Do deep neural networks predict early visual neurons best?
Deep convolutional networks predicted monkey V1 responses to natural images far better than classic Gabor-style models, which suggests V1 computation involves several nonlinear steps.
- Can a predictive map explain planning with simple learning?
Learning values on top of a map of which states tend to follow which lets an agent show flexible, planning-like behaviour using the same prediction-error rule linked to dopamine, and adding offline replay closes the remaining gaps.
- Could brain-like local learning rules approximate backpropagation?
A predictive coding network that changes each synapse using only the activity of the two neurons it connects can learn almost exactly like backpropagation, reaching the same accuracy on handwritten digits.
Compare studies
Select 2–10 studies. Design and N are labels, not a ranking.
Nothing selected yet.
Questions
What is still open
Supervision is essential for IT but not obviously for V1: the IT study found only the supervised network matched, while in V1 a network trained only on neural data (no object labels) came close to VGG (49.8% vs 51.6%) though it needed more data.
Ask PaperFren about Deep networks as models of the brain
Study this conceptflashcards and short-answer questions
What does the 2014 IT study suggest about supervised vs unsupervised models?
Unsupervised and weakly supervised models correlated only weakly with IT and lacked its animate/inanimate split. A deep supervised network's layer 7 matched human IT better, and a reweighted combination reached the noise ceiling. However, only one supervised network with a large training set was tested, so supervision is confounded with architecture and data.
Why is 'explains 51.6% of explainable variance' not the end of the story for V1 models?
It is the best result but still leaves nearly half unexplained. The data are from two monkeys and brief feedforward windows. And CNNs are hard to interpret, so they don't reveal which nonlinearities the cells implement.