Contrastive Decoding: Open-ended Text Generation as Optimization

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 1901

Earlier work this paper cites.

On faithfulness and factuality in abstractive summarization

Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020 · 1919

Earlier work this paper cites.

Logic and conversation

H. Paul Grice. 1975 · 1975

Earlier work this paper cites.

Toward a new taxonomy for pragmatic inference: Q-based and r-based implicature

Laurence Horn. 1984 · 1984

Earlier work this paper cites.

Presumptive Meanings: The Theory of Generalized Conversational Implicature

Stephen C Levinson. 2000 · 2000

Earlier work this paper cites.

Decoding methods for neural narrative generation

Original

Alexandra DeLucia, Aaron Mueller, Xiang Lisa Li, and João Sedoc. 2020 · 2010

Earlier work this paper cites.

Improving multi-step prediction of learned time series models

Arun Venkatraman, Martial Hebert, and J.. Bagnell. 2015 · 2015

Earlier work this paper cites.

Aligning books and movies: Towards story-like visual explanations by watching movies and reading books

Original

Yukun Zhu, Ryan Kiros, Richard Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015 · 2015

Earlier work this paper cites.

Professor forcing: A new algorithm for training recurrent networks

Alex M Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. 2016 · 2016

Earlier work this paper cites.

A diversity-promoting objective function for neural conversation models

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016 · 2016

Earlier work this paper cites.

Sequence level training with recurrent neural networks

Original

Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016 · 2016

Cited alongside, same era.

Sequence-to-sequence learning as beam-search optimization

Sam Wiseman and Alexander M. Rush. 2016 · 2016

Cited alongside, same era.

Google’s neural machine translation system: Bridging the gap between human and machine translation

Original

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016 · 2016

Cited alongside, same era.

Pointer sentinel mixture models

Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017 · 2017

Cited alongside, same era.

Hierarchical neural story generation

Angela Fan, Mike Lewis, and Yann Dauphin. 2018 · 2018

Cited alongside, same era.

Neural text generation with unlikelihood training

Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2020 · 2020

Later among the works it cites.

SimCSE: Simple contrastive learning of sentence embeddings

Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021 · 2021

Later among the works it cites.

Prefix-tuning: Optimizing continuous prompts for generation

Xiang Lisa Li and Percy Liang. 2021 · 2021

Later among the works it cites.

DExperts: Decoding-time controlled text generation with experts and anti-experts

Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021 · 2021

Later among the works it cites.

MAUVE: Measuring the gap between neural text and human text using divergence frontiers

Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021 · 2021

Later among the works it cites.