An Analysis of the Utility of Explicit Negative Examples to Improve the Syntactic Abilities of Neural Language Models

Negative evidence in language acquisition

Gary F. Marcus. 1993 · 1993

Earlier work this paper cites.

Linguistic complexity: Locality of syntactic dependencies

Edward Gibson. 1998 · 1998

Earlier work this paper cites.

A probabilistic earley parser as a psycholinguistic model

John Hale. 2001 · 2001

Earlier work this paper cites.

Feature-rich part-of-speech tagging with a cyclic dependency network

Kristina Toutanova, Dan Klein, Christopher D Manning, and Yoram Singer. 2003 · 2003

Earlier work this paper cites.

Data from eye-tracking corpora as evidence for theories of syntactic processing complexity

Vera Demberg and Frank Keller. 2008 · 2008

Earlier work this paper cites.

Expectation-based syntactic comprehension

Roger Levy. 2008 · 2008

Earlier work this paper cites.

Quantity doesn’t buy quality syntax with neural language models

Original

Marten van Schijndel, Aaron Mueller, and Tal Linzen. 2019 · 2010

Earlier work this paper cites.

Extensions of recurrent neural network language model

Tomas Mikolov, Stefan Kombrink, Lukás Burget, Jan Cernocký, and Sanjeev Khudanpur. 2011 · 2011

Earlier work this paper cites.

Sequence to sequence learning with neural networks

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014 · 2014

Earlier work this paper cites.

Recurrent neural network grammars

Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A. Smith. 2016 · 2016

Cited alongside, same era.

Tying word vectors and word classifiers: A loss framework for language modeling

Hakan Inan, Khashayar Khosravi, and Richard Socher. 2016 · 2016

Cited alongside, same era.

Assessing the ability of lstms to learn syntax-sensitive dependencies

Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016 · 2016

Cited alongside, same era.

Exploring the syntactic abilities of RNNs with multi-task learning

Émile Enguehard, Yoav Goldberg, and Tal Linzen. 2017 · 2017

Cited alongside, same era.

Colorless green recurrent networks dream hierarchically

Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018 · 2018

Cited alongside, same era.

Large margin neural language model

Jiaji Huang, Yi Li, Wei Ping, and Liang Huang. 2018 · 2018

Distinct patterns of syntactic agreement errors in recurrent networks and humans

Tal Linzen and Brian Leonard. 2018 · 2018

Later among the works it cites.

Targeted syntactic evaluation of language models

Rebecca Marvin and Tal Linzen. 2018 · 2018

Later among the works it cites.

Regularizing and optimizing LSTM language models

Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018 · 2018

Later among the works it cites.

The curious case of neural text degeneration

Original

Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019 · 2019

Later among the works it cites.

Scalable syntax-aware language models using knowledge distillation

Adhiguna Kuncoro, Chris Dyer, Laura Rimell, Stephen Clark, and Phil Blunsom. 2019 · 2019

Later among the works it cites.

Studying the inductive biases of RNNs with synthetic variations of natural languages