A Theory of Local Learning, the Learning Channel, and the Optimality of Backpropagation

Understand

In a physical neural system, where storage and processing are intimately intertwined, the rules for adjusting the synaptic weights can only depend on variables that are available locally, such as the activity of the pre- and post-synaptic neurons, resulting in local learning rules.

• A systematic framework for studying the space of local learning rules is obtained by first specifying the nature of the local variables, and then the functional form that ties them together into each learning rule.
• Such a framework enables also the systematic discovery of new learning rules and exploration of relationships between learning rules and group symmetries.
• We study polynomial local learning rules stratified by their degree and analyze their behavior and capabilities in both linear and non-linear units and networks.