Improving Language Modeling using Densely Connected Recurrent Neural Networks

Neural net language models

Yoshua Bengio. 2008 · 2008

Earlier work this paper cites.

Recurrent neural network based language model

Tomas Mikolov, Martin Karafiát, Lukás Burget, Jan Cernocký, and Sanjeev Khudanpur. 2010 · 2010

Earlier work this paper cites.

Context dependent recurrent neural network language model

Tomas Mikolov and Geoffrey Zweig. 2012 · 2012

Earlier work this paper cites.

Sequence to sequence learning with neural networks

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014 · 2014

Earlier work this paper cites.

Recurrent neural network regularization

Original

Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014 · 2014

Earlier work this paper cites.

Finding function in form: Compositional character models for open vocabulary word representation

Wang Ling, Chris Dyer, Alan W. Black, Isabel Trancoso, Ramon Fermandez, Silvio Amir, Luís Marujo, and Tiago Luís. 2015 · 2015

Cited alongside, same era.

A theoretically grounded application of dropout in recurrent neural networks

Yarin Gal and Zoubin Ghahramani. 2016 · 2016

Cited alongside, same era.

Lstm, gru, highway and a bit of attention: an empirical overview for language modeling in speech recognition

Kazuki Irie, Zoltán Tüske, Tamer Alkhouli, Ralf Schlüter, and Hermann Ney. 2016 · 2016

Cited alongside, same era.

Character-aware neural language models

Yoon Kim, Yacine Jernite, David Sontag, and Alexander M. Rush. 2016 · 2016

Cited alongside, same era.

Using the output embedding to improve language models

Original

Ofir Press and Lior Wolf. 2016 · 2016

Cited alongside, same era.

Pixel recurrent neural networks

Aäron Van Den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. 2016 · 2016

Later among the works it cites.

Google’s neural machine translation system: Bridging the gap between human and machine translation

Original

Yonghui Wu, Mike Schuster, and Zhifeng Chen et al. 2016 · 2016

Later among the works it cites.

Densely connected convolutional networks

Gao Huang, Zhuang Liu, Kilian Q Weinberger, and Laurens van der Maaten. 2017 · 2017

Closest in time.

Tying word vectors and word classifiers: A loss framework for language modeling

Hakan Inan, Khashayar Khosravi, and Richard Socher. 2017 · 2017

Closest in time.

Pointer sentinel mixture models

Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017 · 2017

Closest in time.