Fetching the paper…

Reading the bibliography…

2016

Learning Operations on a Stack with Neural Turing Machines

Deleu, Tristan, Dureau, Joseph

Original
PDF
Post on X

Understand

Multiple extensions of Recurrent Neural Networks (RNNs) have been proposed recently to address the difficulty of storing information over long time periods.

In this paper, we experiment with the capacity of Neural Turing Machines (NTMs) to deal with these long-term dependencies on well-balanced strings of parentheses.
We show that not only does the NTM emulate a stack with its heads and learn an algorithm to recognize such words, but it is also capable of strongly generalizing to much longer sequences.

Learning Operations on a Stack with Neural Turing Machines · Around

Built on

Foundations of recurrent neural networks, 1993
Hava (Eve) Tova Siegelmann · 1993
Earlier work this paper cites.
Context-Free Languages and Push-Down Automata
Jean-Michel Autebert, Jean Berstel, and Luc Boasson · 1997
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Enumerative combinatorics. Volume 2
Richard P. Stanley · 1999
Earlier work this paper cites.
LSTM recurrent networks learn simple context-free and context-sensitive languages
Felix A Gers and Jürgen Schmidhuber · 2001
Earlier work this paper cites.
Analytic Combinatorics
Philippe Flajolet and Robert Sedgewick · 2009
Earlier work this paper cites.

Similar

Neural Turing Machines
Original
Alex Graves, Greg Wayne, and Ivo Danihelka · 2014
Cited alongside, same era.
Adam: A Method for Stochastic Optimization
Original
Diederik P. Kingma and Jimmy Ba · 2014
Cited alongside, same era.
Learning to Transduce with Unbounded Memory
Original
Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom · 2015
Cited alongside, same era.
Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
Original
Armand Joulin and Tomas Mikolov · 2015
Cited alongside, same era.
Visualizing and understanding recurrent networks
Original
Andrej Karpathy, Justin Johnson, and Fei-Fei Li · 2015

Then

Weakly Supervised Memory Networks
Original
Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus · 2015
Later among the works it cites.
Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes
Original
Çaglar Gülçehre, Sarath Chandar, Kyunghyun Cho, and Yoshua Bengio · 2016
Closest in time.
Key-Value Memory Networks for Directly Reading Documents
Original
Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston · 2016
Closest in time.
One-shot Learning with Memory-Augmented Neural Networks
Original
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy P. Lillicrap · 2016
Closest in time.
Dynamic Memory Networks for Visual and Textual Question Answering

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Neural Programmer-Interpreters

Scott E. Reed and Nando de Freitas · 2015

Cited alongside, same era.

Caiming Xiong, Stephen Merity, and Richard Socher · 2016

Closest in time.