Fetching the paper…
Reading the bibliography…
Recent work by Hewitt et al.
On the computational power of RNNs
Samuel A. Korsky and Robert C. Berwick. 2019 · 1906
Earlier work this paper cites.
A logical calculus of the ideas immanent in nervous activity
Warren S. McCulloch and Walter Pitts. 1943 · 1943
Earlier work this paper cites.
Neural Nets and the Brain Model Problem
Marvin Lee Minsky. 1954 · 1954
Earlier work this paper cites.
Representation of events in nerve nets and finite automata
S. C. Kleene. 1956 · 1956
Earlier work this paper cites.
Formal language theory: Refining the Chomsky hierarchy
Gerhard Jäger and James Rogers. 2012 · 1970
Earlier work this paper cites.
Threshold matrices and the state assignment problem for neural nets
A. K. Dewdney. 1977 · 1977
Earlier work this paper cites.
Finding structure in time
Jeffrey L. Elman. 1990 · 1990
Earlier work this paper cites.
On the computational power of neural nets
Hava T. Siegelmann and Eduardo D. Sontag. 1992 · 1992
Earlier work this paper cites.
Optimal simulation of automata by neural nets
P. Indyk. 1995 · 1995
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997
Earlier work this paper cites.
Relating probabilistic grammars and automata
Steven Abney, David McAllester, and Fernando Pereira. 1999 · 1999
Earlier work this paper cites.
Predicting sentences using n-gram language models
Steffen Bickel, Peter Haider, and Tobias Scheffer. 2005 · 2005
Earlier work this paper cites.
RNNs can generate bounded hierarchical languages with optimal memory
John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, and Christopher D. Manning. 2020 · 2010
Cited alongside, same era.
Context-free transductions with neural stacks
Yiding Hao, William Merrill, Dana Angluin, Robert Frank, Noah Amsel, Andrew Benz, and Simon Mendelsohn. 2018 · 2018
Cited alongside, same era.
Breaking the softmax bottleneck: A high-rank RNN language model
Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W. Cohen. 2018 · 2018
Cited alongside, same era.
Sequential neural networks as automata
William Merrill. 2019 · 2019
Cited alongside, same era.
A formal hierarchy of RNN architectures
William Merrill, Gail Weiss, Yoav Goldberg, Roy Schwartz, Noah A. Smith, and Eran Yahav. 2020 · 2020
Cited alongside, same era.
Pre-trained models for natural language processing: A survey
XiPeng Qiu, TianXiang Sun, YiGe Xu, YunFan Shao, Ning Dai, and XuanJing Huang. 2020 · 2020
A measure-theoretic characterization of tight language models
Li Du, Lucas Torroba Hennigen, Tiago Pimentel, Clara Meister, Jason Eisner, and Ryan Cotterell. 2023 · 2023
Later among the works it cites.
The surprising computational power of nondeterministic stack RNNs
Brian DuSell and David Chiang. 2023 · 2023
Later among the works it cites.
Formal languages and the NLP black box
William Merrill. 2023 · 2023
Later among the works it cites.
Resurrecting recurrent neural networks for long sequences
Antonio Orvieto, Samuel L Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De. 2023 · 2023
Later among the works it cites.
RWKV: Reinventing RNNs for the transformer era
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Xiangru Tang, Bolun Wang, Johan S. Wind, Stansilaw Wozniak, Ruichong Zhang, Zhenyuan Zhang, Qihang Zhao, Peng Zhou, Jian Zhu, and Rui-Jie Zhu. 2023 · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
fmri reveals language-specific predictive coding during naturalistic sentence comprehension
Cory Shain, Idan Asher Blank, Marten van Schijndel, William Schuler, and Evelina Fedorenko. 2020 · 2020
Cited alongside, same era.
Self-attention networks can process bounded hierarchical languages
Shunyu Yao, Binghui Peng, Christos Papadimitriou, and Karthik Narasimhan. 2021 · 2021
Cited alongside, same era.
Softmax bottleneck makes language models unable to represent multi-mode word distributions
Haw-Shiuan Chang and Andrew McCallum. 2022 · 2022
Cited alongside, same era.
Saturated transformers are constant-depth threshold circuits
William Merrill, Ashish Sabharwal, and Noah A. Smith. 2022 · 2022
Cited alongside, same era.
Extracting finite automata from RNNs using state merging
William Merrill and Nikolaos Tsilivis. 2022 · 2022
Cited alongside, same era.
Efficiently representing finite-state automata with recurrent neural networks
Anej Svete and Ryan Cotterell. 2023a
Cited in the paper.
Later among the works it cites.
Transformers as recognizers of formal languages: A survey on expressivity
Lena Strobl, William Merrill, Gail Weiss, David Chiang, and Dana Angluin. 2023 · 2023
Later among the works it cites.
RecurrentGPT: Interactive generation of (arbitrarily) long text
Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, and Mrinmaya Sachan. 2023 · 2023
Later among the works it cites.
What languages are easy to language-model? a perspective from learning probabilistic regular languages
Nadav Borenstein, Anej Svete, Robin Shing Moon Chan, Josef Valvoda, Franz Nowak, Isabelle Augenstein, Eleanor Chodroff, and Ryan Cotterell. 2024 · 2024
Closest in time.
Formal aspects of language modeling
Ryan Cotterell, Anej Svete, Clara Meister, Tianyu Liu, and Li Du. 2024 · 2024
Closest in time.
Lower bounds on the expressivity of recurrent neural language models
Anej Svete, Franz Nowak, Anisha Mohamed Sahabdeen, and Ryan Cotterell. 2024 · 2024
Closest in time.
Computational models to study language processing in the human brain: A survey
Shaonan Wang, Jingyuan Sun, Yunhao Zhang, Nan Lin, Marie-Francine Moens, and Chengqing Zong. 2024 · 2024
Closest in time.