Neural Turing Machines: Convergence of Copy Tasks

Understand

The architecture of neural Turing machines is differentiable end to end and is trainable with gradient descent methods.

• Due to their large unfolded depth Neural Turing Machines are hard to train and because of their linear access of complete memory they do not scale.
• Other architectures have been studied to overcome these difficulties.
• In this report we focus on improving the quality of prediction of the original linear memory architecture on copy and repeat copy tasks.