Fetching the paper…

Reading the bibliography…

Generating News Headlines with Recurrent Neural Networks · Around

2015

Generating News Headlines with Recurrent Neural Networks

Lopyrev, Konstantin

Original
PDF
Post on X

Understand

We describe an application of an encoder-decoder recurrent neural network with LSTM units and attention to generating headlines from the text of news articles.

We find that the model is quite effective at concisely paraphrasing news articles.
Furthermore, we study how the neural network decides which input words to pay attention to, and specifically we identify the function of the different neurons in a simplified attention mechanism.
Interestingly, our simplified attention mechanism performs better that the more complex attention mechanism on a held out set of articles.

Built on

Bleu: A method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu · 2002
Earlier work this paper cites.
Lecture 6.5 - rmsprop: Divide the gradient by a running average of its recent magnitude
Tijmen Tieleman and Geoffrey Hinton · 2012
Earlier work this paper cites.
Speech recognition with deep recurrent neural networks
Original
Alex Graves, Abdel-rahman Mohamed, and Geoffrey E. Hinton · 2013
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Original
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le · 2014
Earlier work this paper cites.

Similar

Recurrent neural network regularization
Original
Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals · 2014
Cited alongside, same era.
Deep visual-semantic alignments for generating image descriptions
Original
Andrej Karpathy and Fei-Fei Li · 2014
Cited alongside, same era.
Effective approaches to attention-based neural machine translation
Original
Minh-Thang Luong, Hieu Pham, and Christopher D. Manning · 2015
Cited alongside, same era.
Teaching machines to read and comprehend
Original
Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom · 2015
Cited alongside, same era.

Then

A neural attention model for abstractive sentence summarization
Original
Alexander M. Rush, Sumit Chopra, and Jason Weston · 2015
Closest in time.
Deep learning
Ian Goodfellow, Aaron Courville, and Yoshua Bengio · 2015
Closest in time.
Scheduled sampling for sequence prediction with recurrent neural networks
Original
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer · 2015
Closest in time.
Listen, attend and spell
Original
William Chan, Navdeep Jaitly, Quoc V. Le, and Oriol Vinyals · 2015
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…