Memory and attention in deep learning

Understand

Intelligence necessitates memory.

• Without memory, humans fail to perform various nontrivial tasks such as reading novels, playing games or solving maths.
• As the ultimate goal of machine learning is to derive intelligent systems that learn and act automatically just like human, memory construction for machine is inevitable.
• Artificial neural networks model neurons and synapses in the brain by interconnecting computational units via weights, which is a typical class of machine learning algorithms that resembles memory structure.