Learning to Insert [PAUSE] Tokens for Better Reasoning

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Augmenting self-attention with persistent memory

Original

Sainbayar Sukhbaatar, Edouard Grave, Guillaume Lample, Herve Jegou, and Armand Joulin. 2019 · 1907

Earlier work this paper cites.

CTRL - A Conditional Transformer Language Model for Controllable Generation

Original

Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. 2019 · 1909

Earlier work this paper cites.

Scaling laws for neural language models

Original

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020 · 2001

Earlier work this paper cites.

Greedy decoding for statistical machine translation in almost linear time

Ulrich Germann. 2003 · 2003

Earlier work this paper cites.

Tldr: token loss dynamic reweighting for reducing repetitive utterance generation

Original

Shaojie Jiang, Thomas Wolf, Christof Monz, and Maarten de Rijke. 2020 · 2003

Earlier work this paper cites.

Memory transformer

Original

Mikhail S Burtsev, Yuri Kuratov, Anton Peganov, and Grigory V Sapunov. 2020 · 2006

Earlier work this paper cites.

Posterior calibration and exploratory analysis for natural language processing models

Khanh Nguyen and Brendan O’Connor. 2015 · 2015

Earlier work this paper cites.

Beam search strategies for neural machine translation

Markus Freitag and Yaser Al-Onaizan. 2017 · 2017

Earlier work this paper cites.

Focal loss for dense object detection

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017 · 2017

Earlier work this paper cites.

Program induction by rationale generation: Learning to solve and explain algebraic word problems

Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Universal transformers

Original

Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. 2018 · 2018

Earlier work this paper cites.

Confidence modeling for neural semantic parsing

Li Dong, Chris Quirk, and Mirella Lapata. 2018 · 2018

Earlier work this paper cites.

Improving language understanding by generative pre-training

Alec Radford and Karthik Narasimhan. 2018 · 2018

Earlier work this paper cites.

Improving language understanding by generative pre-training

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018 · 2018

Earlier work this paper cites.

End-to-end bias mitigation by modelling biases in corpora

Rabeeh Karimi Mahabadi, Yonatan Belinkov, and James Henderson. 2020 · 2020

Cited alongside, same era.

Calibrating deep neural networks using focal loss

Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr, and Puneet Dokania. 2020 · 2020

Cited alongside, same era.

Program synthesis with large language models

Original

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021 · 2021

Cited alongside, same era.

Training verifiers to solve math word problems

Original

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021 · 2021

Cited alongside, same era.

Dynamically weighted balanced loss: class imbalanced learning and confidence calibration of deep neural networks

K Ruwani M Fernando and Chris P Tsokos. 2021 · 2021

Fast inference from transformers via speculative decoding

Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023 · 2023

Later among the works it cites.

Phi-2: The surprising power of small language models

Microsoft. 2023 · 2023

Later among the works it cites.

From words to watts: Benchmarking the energy costs of large language model inference

Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. 2023 · 2023

Later among the works it cites.

Dual focal loss for calibration

Linwei Tao, Minjing Dong, and Chang Xu. 2023 · 2023

Later among the works it cites.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023

Later among the works it cites.