Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models

Understand

A large number of neural network models of associative memory have been proposed in the literature.

• These include the classical Hopfield networks (HNs), sparse distributed memories (SDMs), and more recently the modern continuous Hopfield networks (MCHNs), which possesses close links with self-attention in machine learning.
• In this paper, we propose a general framework for understanding the operation of such memory networks as a sequence of three operations: similarity, separation, and projection.
• We derive all these memory models as instances of our general framework with differing similarity and separation functions.