Spikingformer: A Key Foundation Model for Spiking Neural Networks

Understand

Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks, due to their event-driven spiking computation.

• However, some foundation SNN backbones (including Spikformer and SEW ResNet) suffer from non-spike computations (integer-float multiplications) caused by the structure of their residual connections.
• These non-spike computations increase SNNs' power consumption and make them unsuitable for deployment on mainstream neuromorphic hardware.
• In this paper, we analyze the spike-driven behavior of the residual connection methods in SNNs.