Elucidating the theoretical underpinnings of surrogate gradient learning in spiking neural networks

Understand

Training spiking neural networks to approximate universal functions is essential for studying information processing in the brain and for neuromorphic computing.

• Yet the binary nature of spikes poses a challenge for direct gradient-based training.
• Surrogate gradients have been empirically successful in circumventing this problem, but their theoretical foundation remains elusive.
• Here, we investigate the relation of surrogate gradients to two theoretically well-founded approaches.