Deep Predictive Coding Networks

Understand

The quality of data representation in deep learning methods is directly related to the prior model imposed on the representations; however, generally used fixed priors are not capable of adjusting to the context in the data.

• To address this issue, we propose deep predictive coding networks, a hierarchical generative model that empirically alters priors on the latent representations in a dynamic and context-sensitive manner.
• This model captures the temporal dependencies in time-varying signals and uses top-down information to modulate the representation in lower layers.
• The centerpiece of our model is a novel procedure to infer sparse states of a dynamic model which is used for feature extraction.