Training Neural Networks with Local Error Signals

Understand

Supervised training of neural networks for classification is typically performed with a global loss function.

• The loss function provides a gradient for the output layer, and this gradient is back-propagated to hidden layers to dictate an update direction for the weights.
• An alternative approach is to train the network with layer-wise loss functions.
• In this paper we demonstrate, for the first time, that layer-wise training can approach the state-of-the-art on a variety of image datasets.