Can spiking neural networks learn precise timing with hidden layers?
A surrogate-gradient rule lets multilayer spiking networks learn precisely timed outputs, but on hard tasks random feedback does much worse than backprop-like symmetric feedback.
Source
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks
Study at a glance
- Design
- Computational / modelling — Simulations of leaky integrate-and-fire networks trained with the SuperSpike rule under three feedback schemes on synthetic spike-pattern tasks
- N
- No dataset N; tasks are synthetic (single neuron with 100 inputs, a spiking XOR task, and a 100-output pattern task)
- Population
- Simulated spiking neural networks
- Outcome
- Van Rossum distance to target spike trains and classification accuracy
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors derived SuperSpike, a learning rule that replaces the non-differentiable spike with a smooth function of membrane voltage, giving a local three-factor Hebbian update with eligibility traces. They trained simulated leaky integrate-and-fire networks to produce target spike trains and to solve a spiking exclusive-or task. Hidden units received error signals via symmetric (backprop-like), random (feedback alignment) or uniform (global) feedback.
What they found
A single neuron that started completely silent learned a five-spike target after 500 trials, without injecting noise. On the spiking XOR task, networks with hidden layers learned it perfectly while a network without hidden units could not; uniform feedback was worse on average. On a harder task with 100 output neurons, symmetric feedback with 32 or more hidden units matched the target visually, while random feedback performed worse than even a network with no hidden layer, partly improved by an activity regulariser.
The limits
What it doesn't show
All tasks are small synthetic spike-pattern problems, not real-world benchmarks, so it says little about scaling to large datasets. The eligibility traces are computed per synapse, so cost grows with the square of neuron count, which the authors flag. Biological interpretations (calcium transients, neuromodulators) are suggestions, not tested against neural data. Only shallow three-layer feedforward networks were studied, not deep or recurrent ones.
Key terms
- Spiking neural network
- A network whose units communicate by discrete spikes in time rather than continuous activation values.
- Surrogate gradient
- A smooth stand-in derivative used in place of the true, zero-almost-everywhere derivative of a spike so gradient descent can work.
- Van Rossum distance
- A measure of difference between two spike trains computed after smoothing each with a temporal kernel.
- Feedback alignment
- Sending errors to hidden units through fixed random weights instead of the transpose of the forward weights.
- Three-factor rule
- A plasticity rule combining presynaptic activity, postsynaptic activity and a third error or modulatory signal.
Flashcards
0 of 11 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What does SuperSpike substitute for the spike train when computing gradients?
Common questions
Why can't normal backpropagation train spiking neurons directly?
A spike is an all-or-nothing event, so its derivative is zero almost everywhere and undefined at the spike, giving no useful gradient; SuperSpike substitutes a smooth function of voltage.
Does random feedback work as well as backprop here?
On simple tasks it did, but on the harder 100-output timing task it was clearly worse than symmetric feedback, even with regularisation.
What is special about starting from a silent network?
Earlier methods needed noise to make neurons fire before learning could begin; SuperSpike's voltage-based gradient works even when hidden units never spike initially.