Built on

Hava T. Siegelmann , Eduardo D. Sontag. ”On the computational power of neural nets.” Proceedings of the fifth annual workshop on Computational learning theory, p.440-449, July 27-29, 1992, Pittsburgh, Pennsylvania, USA http://binds.cs.umass.edu/papers/1992_Siegelmann_COLT.pdf
1992
Earlier work this paper cites.
Autebert, Jean-Michel; Berstel, Jean; Boasson, Luc (1997). ”Context-Free Languages and Push-Down Automata” (PDF). In G. Rozenberg and A. Salomaa, eds., Handbook of Formal Languages, Vol. 1: Word, Language, Grammar (pp. 111–174). Berlin: Springer-Verlag. ISBN 3-540-60420-0
Original
1997
Earlier work this paper cites.
Gers, Felix A. and Jürgen Schmidhuber. “LSTM recurrent networks learn simple context-free and context-sensitive languages.” IEEE transactions on neural networks 12 6 (2001): 1333-40. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=963769
2001
Earlier work this paper cites.
Sundermeyer, Martin et al. “LSTM Neural Networks for Language Modeling.” INTERSPEECH (2012). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.248.4448&rep=rep1&type=pdf
2012
Earlier work this paper cites.

Similar

Wen, Tsung-Hsien et al. “Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems.” EMNLP (2015). https://arxiv.org/abs/1508.01745
Original
2015
Cited alongside, same era.
Chiu, Jason P. C. and Eric Nichols. “Named Entity Recognition with Bidirectional LSTM-CNNs.” TACL 4 (2016): 357-370. https://www.aclweb.org/anthology/Q16-1026
2016
Cited alongside, same era.
Ke Tran, Arianna Bisazza, and Christof Monz. ”The importance of being recurrent for modeling hierarchical structure.” In Proceedings of NAACL’18, 2018. https://arxiv.org/pdf/1803.03585.pdf
Original
2018
Cited alongside, same era.
Zhang, Lei et al. “Deep learning for sentiment analysis: A survey.” Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 8 (2018): n. pag. https://arxiv.org/ftp/arxiv/papers/1801/1801.07883.pdf
2018
Cited alongside, same era.

Then

Yining Chen, Sorcha Gilroy, Kevin Knight, and Jonathan May. 2018. Recurrent neural networks as weighted language recognizers. In Proc. of NAACL. https://arxiv.org/pdf/1711.05408.pdf
Original
2018
Later among the works it cites.
Weiss, Gail et al. “On the Practical Computational Power of Finite Precision RNNs for Language Recognition.” ACL (2018). https://arxiv.org/pdf/1805.04908.pdf
Original
2018
Later among the works it cites.
Jean-Phillippe Bernardy. 2018. Can Recurrent Neural Networks Learn Nested Recursion? Linguistic Issues in Language Technology, 16(1)
2018
Later among the works it cites.
Skachkova, Natalia et al. ”Closing Brackets with Recurrent Neural Networks.” ACL (2018) http://aclweb.org/anthology/W18-5425
2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2019

On the Computational Power of RNNs

Korsky, Samuel A., Berwick, Robert C.

Understand

Recent neural network architectures such as the basic recurrent neural network (RNN) and Gated Recurrent Unit (GRU) have gained prominence as end-to-end learning architectures for natural language processing tasks.

But what is the computational power of such systems? We prove that finite precision RNNs with one hidden layer and ReLU activation and finite precision GRUs are exactly as computationally powerful as deterministic finite automata.
Allowing arbitrary precision, we prove that RNNs with one hidden layer and ReLU activation are at least as computationally powerful as pushdown automata.
If we also allow infinite precision, infinite edge weights, and nonlinear output activation functions, we prove that GRUs are at least as computationally powerful as pushdown automata.