Predictive Coding Approximates Backprop along Arbitrary Computation Graphs

Understand

Backpropagation of error (backprop) is a powerful algorithm for training machine learning architectures through end-to-end differentiation.

• However, backprop is often criticised for lacking biological plausibility.
• Recently, it has been shown that backprop in multilayer-perceptrons (MLPs) can be approximated using predictive coding, a biologically-plausible process theory of cortical computation which relies only on local and Hebbian updates.
• The power of backprop, however, lies not in its instantiation in MLPs, but rather in the concept of automatic differentiation which allows for the optimisation of any differentiable program expressed as a computation graph.