Fetching the paper…
Reading the bibliography…
Since language models are used to model a wide variety of languages, it is natural to ask whether the neural architectures used for the task have inductive biases towards modeling particular types of languages.
Some universals of grammar with particular reference to the order of meaningful elements
Joseph H. Greenberg. 1963 · 1963
Earlier work this paper cites.
Lectures on Government and Binding: The Pisa Lectures
Noam Chomsky. 1981 · 1981
Earlier work this paper cites.
Long Short-Term Memory
Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997
Earlier work this paper cites.
Competitive grammar writing
Jason Eisner and Noah A. Smith. 2008 · 2008
Earlier work this paper cites.
Mixed Effects Models for Complex Data
Lang Wu. 2009 · 2009
Earlier work this paper cites.
Learning biases predict a word order universal
Jennifer Culbertson, Paul Smolensky, and Géraldine Legendre. 2012 · 2012
Earlier work this paper cites.
LSTM neural networks for language modeling
Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. 2012 · 2012
Earlier work this paper cites.
Order of subject, object and verb
Matthew S. Dryer. 2013 · 2013
Cited alongside, same era.
WALS Online
Matthew S. Dryer and Martin Haspelmath, editors. 2013 · 2013
Cited alongside, same era.
Taming the Jabberwocky: Examining Sentence Processing with Novel Words
Gaurav Kharkwal. 2014 · 2014
Cited alongside, same era.
Tree-structured composition in neural networks without tree-structured architectures
Samuel R. Bowman, Christopher D. Manning, and Christopher Potts. 2015 · 2015
Cited alongside, same era.
The galactic dependencies treebanks: Getting more data by synthesizing new languages
Dingquan Wang and Jason Eisner. 2016 · 2016
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.
Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks
Brenden Lake and Marco Baroni. 2018 · 2018
Later among the works it cites.
From the world to word order: the link between conceptual structure and language
Jennifer Culbertson, Marieke Schouwstra, and Simon Kirby. 2019 · 2019
Later among the works it cites.
What kind of language is hard to language-model?
Sabrina J. Mielke, Ryan Cotterell, Kyle Gorman, Brian Roark, and Jason Eisner. 2019 · 2019
Later among the works it cites.
fairseq: A fast, extensible toolkit for sequence modeling
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019 · 2019
Later among the works it cites.
Studying the inductive biases of RNNs with synthetic variations of natural languages
Shauli Ravfogel, Yoav Goldberg, and Tal Linzen. 2019 · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Are all languages equally hard to language-model?
Ryan Cotterell, Sabrina J. Mielke, Jason Eisner, and Brian Roark. 2018 · 2018
Cited alongside, same era.
Revisiting the poverty of the stimulus: Hierarchical generalization without a hierarchical bias in recurrent neural networks
R. Thomas McCoy, Robert Frank, and Tal Linzen. 2018 · 2098
Closest in time.