2016

Recurrent Neural Network Grammars

Dyer, Chris, Kuncoro, Adhiguna, Ballesteros, Miguel et al.

Understand

We introduce recurrent neural network grammars, probabilistic models of sentences with explicit phrase structure.

We explain efficient inference procedures that allow application to both parsing and language modeling.
Experiments show that they provide better parsing in English than any single previously published supervised generative model and better language modeling than state-of-the-art sequential RNNs in English and Chinese.

Built on

Syntactic Structures
Noam Chomsky. 1957 · 1957
Earlier work this paper cites.
An efficient context-free parsing algorithm
Jay Earley. 1970 · 1970
Earlier work this paper cites.
Trainable grammars for speech recognition
James K. Baker. 1979 · 1979
Earlier work this paper cites.
Finding structure in time
Jeffrey L. Elman. 1990 · 1990
Earlier work this paper cites.
Computation of the probability of initial substring generation by stochastic context-free grammars
Frederick Jelinek and John D. Lafferty. 1991 · 1991
Earlier work this paper cites.
Class-based n-gram models of natural language
Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. 1992 · 1992
Earlier work this paper cites.
On the computational power of neural nets
Hava T. Siegelmann and Eduardo D. Sontag. 1995 · 1995
Earlier work this paper cites.
The neural network pushdown automaton: Architecture, dynamics and training
Guo-Zheng Sun, C. Lee Giles, and Hsing-Hen Chen. 1998 · 1998
Earlier work this paper cites.
Relating probabilistic grammars and automata
Steven Abney, David McAllester, and Fernando Pereira. 1999 · 1999
Earlier work this paper cites.
A maximum-entropy-inspired parser
Eugene Charniak. 2000 · 2000
Earlier work this paper cites.
Structured language modeling
Ciprian Chelba and Frederick Jelinek. 2000 · 2000
Earlier work this paper cites.
Classes for fast maximum entropy training
Joshua Goodman. 2001 · 2001
Earlier work this paper cites.
Joint and conditional estimation of tagging and parsing models
Mark Johnson. 2001 · 2001
Earlier work this paper cites.
Probabilistic top-down parsing and language modeling
Brian Roark. 2001 · 2001
Earlier work this paper cites.
An efficient implementation of a new DOP model
Rens Bod. 2003 · 2003
Earlier work this paper cites.
Inducing history representations for broad coverage statistical parsing
James Henderson. 2003 · 2003
Earlier work this paper cites.
Feature-rich part-of-speech tagging with a cyclic dependency network
Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003 · 2003
Earlier work this paper cites.
On the parameter space of generative lexicalized statistical parsing models
Dan Bikel. 2004 · 2004
Earlier work this paper cites.

Similar

Discriminative training of a neural network statistical parser
James Henderson. 2004 · 2004
Cited alongside, same era.
Robust garden path parsing
Brian Roark. 2004 · 2004
Cited alongside, same era.
A neural syntactic language model
Ahmad Emami and Frederick Jelinek. 2005 · 2005
Cited alongside, same era.
A classifier-based parser with linear run-time complexity
Kenji Sagae and Alon Lavie. 2005 · 2005
Cited alongside, same era.
The Penn Chinese TreeBank: Phrase structure annotation of a large corpus
Naiwen Xue, Fei Xia, Fu-dong Chiou, and Marta Palmer. 2005 · 2005
Cited alongside, same era.
A fast unified model for parsing and sentence understanding
Original
Samuel R. Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D. Manning, and Christopher Potts. 2006 · 2006
Cited alongside, same era.
Scalable inference and training of context-rich syntactic translation models
Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Ignacio Thayer. 2006 · 2006
Cited alongside, same era.
Effective self-training for parsing
David McClosky, Eugene Charniak, and Mark Johnson. 2006 · 2006
Cited alongside, same era.
Improved inference for unlexicalized parsing
Slav Petrov and Dan Klein. 2007 · 2007
Cited alongside, same era.
A latent variable model for generative dependency parsing
Ivan Titov and James Henderson. 2007 · 2007
Cited alongside, same era.
Self-training PCFG grammars with latent annotations across languages
Zhongqiang Huang and Mary Harper. 2009 · 2009
Cited alongside, same era.
Top-down nearly-context-sensitive parsing
Eugene Charniak. 2010 · 2010
Cited alongside, same era.

Then

Syntactic processing using the generalized perceptron and beam search
Yue Zhang and Stephen Clark. 2011 · 2011
Later among the works it cites.
Bayesian symbol-refined tree substitution grammars for syntactic parsing
Hiroyuki Shindo, Yusuke Miyao, Akinori Fujino, and Masaaki Nagata. 2012 · 2012
Later among the works it cites.
Efficient higher-order CRFs for morphological tagging
Thomas Mueller, Helmut Schmid, and Hinrich Schütze. 2013 · 2013
Later among the works it cites.
Fast and accurate shift-reduce constituent parsing
Muhua Zhu, Yue Zhang, Wenliang Chen, Min Zhang, and Jingbo Zhu. 2013 · 2013
Later among the works it cites.
Learning phrase representations using rnn encoder–decoder for statistical machine translation
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014 · 2014
Later among the works it cites.
Phrase dependency machine translation with quasi-synchronous tree-to-tree features
Kevin Gimpel and Noah A. Smith. 2014 · 2014
Later among the works it cites.
Joint POS tagging and transition-based constituent parsing in Chinese with non-local features
Zhiguo Wang and Nianwen Xue. 2014 · 2014
Later among the works it cites.
Neural machine transation by jointly learning to align and translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015 · 2015
Later among the works it cites.
Pragmatic neural modelling in machine translation
Paul Baltescu and Phil Blunsom. 2015 · 2015
Later among the works it cites.
Transition-based dependency parsing with stack long short-term memory
Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015 · 2015
Later among the works it cites.
Improved semantic representations from tree-structured long short-term memory networks
Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015 · 2015
Later among the works it cites.
Grammar as a foreign language
Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015 · 2015
Later among the works it cites.
Feature optimization for constituent parsing via neural networks
Zhiguo Wang, Haitao Mi, and Nianwen Xue. 2015 · 2015
Later among the works it cites.
Semantically conditioned LSTM-based natural language generation for spoken dialogue systems
Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, Pei-Hao Su, David Vandyke, and Steve Young. 2015 · 2015
Later among the works it cites.
Show, attend and tell: Neural image caption generation with visual attention
Kelvin Xu, Jimmy Lei Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015 · 2015
Later among the works it cites.
Recurrent neural network regularization
Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2015 · 2015
Later among the works it cites.
Easy-first dependency parsing with hierarchical tree LSTMs
Original
Eliyahu Kiperwasser and Yoav Goldberg. 2016 · 2016
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…