Fetching the paper…
Reading the bibliography…
This report is a survey of the relationships between various state-of-the-art neural network architectures and formal languages as, for example, structured by the Chomsky Language Hierarchy.
Counter machines and counter languages
Patrick C Fischer, Albert R Meyer, and Arnold L Rosenberg · 1968
Earlier work this paper cites.
Finding structure in time
Jeffrey L. Elman · 1990
Earlier work this paper cites.
Learning context-free grammars: Capabilities and limitations of a recurrent neural network with an external stack memory
Sreerupa Das, C Lee Giles, and Guo-Zheng Sun · 1992
Earlier work this paper cites.
Analog computation via neural networks
Hava T Siegelmann and Eduardo D Sontag · 1994
Earlier work this paper cites.
A recurrent network that performs a context-sensitive prediction task
Mark Steijvers and Peter Grünwald · 1996
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Designing a counter: Another case study of dynamics and activation landscapes in recurrent networks
Steffen Hölldobler, Yvonne Kalinke, and Helko Lehmann · 1997
Earlier work this paper cites.
Learning a context-free task with a recurrent neural network: An analysis of stability
Bradley Tonkes and Janet Wiles · 1997
Earlier work this paper cites.
Recurrent neural networks can learn to implement symbol-sensitive counting
Paul Rodriguez and Janet Wiles · 1998
Earlier work this paper cites.
Formal language theory: refining the Chomsky hierarchy
Gerhard Jäger and James Rogers · 2012
Cited alongside, same era.
Learning phrase representations using RNN encoder-decoder for statistical machine translation
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio · 2014
Cited alongside, same era.
Generative adversarial networks
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio · 2014
Cited alongside, same era.
Alex Graves, Greg Wayne, and Ivo Danihelka · 2014
Cited alongside, same era.
Lie access neural Turing machine
Greg Yang · 2016
Later among the works it cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Later among the works it cites.
Comparative study of CNN and RNN for natural language processing
Wenpeng Yin, Katharina Kann, Mo Yu, and Hinrich Schütze · 2017
Later among the works it cites.
Dynamic neural Turing machine with continuous and discrete addressing schemes
Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho, and Yoshua Bengio · 2018
Later among the works it cites.
Improving language understanding by generative pre-training
Alec Radford · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom · 2015
Cited alongside, same era.
Inferring algorithmic patterns with stack-augmented recurrent nets
Armand Joulin and Tomas Mikolov · 2015
Cited alongside, same era.
Neural random access machines
Karol Kurach, Marcin Andrychowicz, and Ilya Sutskever · 2015
Cited alongside, same era.
Reinforcement learning neural Turing machines
Wojciech Zaremba and Ilya Sutskever · 2015
Cited alongside, same era.
Hybrid computing using a neural network with dynamic external memory
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen. King, C. Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis · 2016
Cited alongside, same era.
Gail Weiss, Yoav Goldberg, and Eran Yahav · 2018
Later among the works it cites.
Sequential neural networks as automata
William Merrill · 2019
Later among the works it cites.
Memory-augmented recurrent neural networks can learn generalized Dyck languages
Mirac Suzgun, Sebastian Gehrmann, Yonatan Belinkov, and Stuart M. Shieber · 2019
Later among the works it cites.