PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding

Understand

We present PredProp, a method for optimization of weights and states in predictive coding networks (PCNs) based on the precision of propagated errors and neural activity.

• PredProp jointly addresses inference and learning via stochastic gradient descent and adaptively weights parameter updates by approximate curvature.
• Due to the relation between propagated error covariance and the Fisher information matrix, PredProp implements approximate Natural Gradient Descent.
• We demonstrate PredProp's effectiveness in the context of dense decoder networks and simple image benchmark datasets.