Improving Diversity of Neural Text Generation via Inverse Probability Weighting

Huggingface’s transformers: State-of-the-art natural language processing

Original

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2019 · 1910

Earlier work this paper cites.

Power laws, pareto distributions and zipf’s law

Mark EJ Newman. 2005 · 2005

Earlier work this paper cites.

Information Theory: Coding Theorems for Discrete Memoryless Systems , 2 edition

Imre Csiszár and János Körner. 2011 · 2011

Earlier work this paper cites.

Human Behaviour and the Principle of Least Effort

Original

George K. Zipf. 1949 · 2011

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Hierarchical neural story generation

Angela Fan, Mike Lewis, and Yann Dauphin. 2018 · 2018

Cited alongside, same era.

Learning to write with cooperative discriminators

Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. 2018 · 2018

Cited alongside, same era.

Texygen: A benchmarking platform for text generation models

Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018 · 2018

Cited alongside, same era.

Language models are unsupervised multitask learners

Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019

Cited alongside, same era.

Causal Inference: What If

Hernán MA, Robins JM. 2020 · 2020

Later among the works it cites.

The curious case of neural text degeneration

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020 · 2020

Later among the works it cites.

Improved natural language generation via loss truncation

Daniel Kang and Tatsunori Hashimoto. 2020 · 2020

Later among the works it cites.

MIROSTAT: A neural text decoding algorithm that directly controls perplexity

Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, and Lav R. Varshney. 2021 · 2021

Closest in time.