A Theoretical Framework for Target Propagation

Understand

The success of deep learning, a brain-inspired form of AI, has sparked interest in understanding how the brain could similarly learn across multiple layers of neurons.

• However, the majority of biologically-plausible learning algorithms have not yet reached the performance of backpropagation (BP), nor are they built on strong theoretical foundations.
• Here, we analyze target propagation (TP), a popular but not yet fully understood alternative to BP, from the standpoint of mathematical optimization.
• Our theory shows that TP is closely related to Gauss-Newton optimization and thus substantially differs from BP.