Direct Feedback Alignment with Sparse Connections for Local Learning

Understand

Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent.

• Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks.
• Commonly referred to as the weight transport problem, each neuron's dependence on the weights and errors located deeper in the network require exhaustive data movement which presents a key problem in enhancing the performance and energy-efficiency of machine-learning hardware.
• In this work, we propose a bio-plausible alternative to backpropagation drawing from advances in feedback alignment algorithms in which the error computation at a single synapse reduces to the product of three scalar values.