Learning to Remember, Forget and Ignore using Attention Control in Memory

Understand

Typical neural networks with external memory do not effectively separate capacity for episodic and working memory as is required for reasoning in humans.

• Applying knowledge gained from psychological studies, we designed a new model called Differentiable Working Memory (DWM) in order to specifically emulate human working memory.
• As it shows the same functional characteristics as working memory, it robustly learns psychology inspired tasks and converges faster than comparable state-of-the-art models.
• Moreover, the DWM model successfully generalizes to sequences two orders of magnitude longer than the ones used in training.