Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?

Understand

The backpropagation of error algorithm used to train deep neural networks has been fundamental to the successes of deep learning.

• However, it requires sequential backward updates and non-local computations, which make it challenging to parallelize at scale and is unlike how learning works in the brain.
• Neuroscience-inspired learning algorithms, however, such as \emph{predictive coding}, which utilize local learning, have the potential to overcome these limitations and advance beyond current deep learning technologies.
• While predictive coding originated in theoretical neuroscience as a model of information processing in the cortex, recent work has developed the idea into a general-purpose algorithm able to train neural networks using only local computations.