Built on

A learning algorithm for boltzmann machines
David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski · 1985
Earlier work this paper cites.
Gaussian error linear units (gelus)
Original
Dan Hendrycks and Kevin Gimpel · 2016
Earlier work this paper cites.
Pointer sentinel mixture models
Original
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher · 2016
Earlier work this paper cites.
Beam search strategies for neural machine translation
Original
Markus Freitag and Yaser Al-Onaizan · 2017
Earlier work this paper cites.
Attention is all you need
A Vaswani · 2017
Earlier work this paper cites.
Hierarchical neural story generation
Original
Angela Fan, Mike Lewis, and Yann Dauphin · 2018
Earlier work this paper cites.
The curious case of neural text degeneration
Original
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi · 2019
Earlier work this paper cites.
Calibration of encoder decoder models for neural machine translation
Original
Aviral Kumar and Sunita Sarawagi · 2019
Earlier work this paper cites.
Neural text generation with unlikelihood training
Original
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston · 2019
Earlier work this paper cites.
Fine-tuning language models from human preferences
Original
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving · 2019
Earlier work this paper cites.
Mirostat: A neural text decoding algorithm that directly controls perplexity, 2020
Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, and Lav R. Varshney · 2020
Earlier work this paper cites.

Similar

Trading off diversity and quality in natural language generation
Original
Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan · 2020
Cited alongside, same era.
Training verifiers to solve math word problems
Original
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al · 2021
Cited alongside, same era.
Large language models are zero-shot reasoners
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa · 2022
Cited alongside, same era.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Cited alongside, same era.
Self-consistency improves chain of thought reasoning in language models
Original

Then

Some things are more cringe than others: Preference optimization with the pairwise cringe loss
Original
Jing Xu, Andrew Lee, Sainbayar Sukhbaatar, and Jason Weston · 2023
Later among the works it cites.
Improving code generation by dynamic temperature sampling
Yuqi Zhu, Jia Li, Ge Li, YunFei Zhao, Jia Li, Zhi Jin, and Hong Mei · 2023
Later among the works it cites.
The llama 3 herd of models
Original
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al · 2024
Closest in time.
Dynamic stochastic decoding strategy for open-domain dialogue generation
Original
Yiwei Li, Fei Mi, Yitong Li, Yasheng Wang, Bin Sun, Shaoxiong Feng, and Kan Li · 2024
Closest in time.
Simpo: Simple preference optimization with a reference-free reward

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Adaptive Decoding via Latent Preference Optimization

Dhuliawala, Shehzaad, Kulikov, Ilia, Yu, Ping et al.

Understand

During language model decoding, it is known that using higher temperature sampling gives more creative responses, while lower temperatures are more factually accurate.

However, such models are commonly applied to general instruction following, which involves both creative and fact seeking tasks, using a single fixed temperature across all examples and tokens.
In this work, we introduce Adaptive Decoding, a layer added to the model to select the sampling temperature dynamically at inference time, at either the token or example level, in order to optimize performance.
To learn its parameters we introduce Latent Preference Optimization (LPO) a general approach to train discrete latent variables such as choices of temperature.

Built on

A learning algorithm for boltzmann machines
David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski · 1985
Earlier work this paper cites.
Gaussian error linear units (gelus)
Original
Dan Hendrycks and Kevin Gimpel · 2016
Earlier work this paper cites.
Pointer sentinel mixture models
Original
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher · 2016
Earlier work this paper cites.
Beam search strategies for neural machine translation
Original
Markus Freitag and Yaser Al-Onaizan · 2017
Earlier work this paper cites.
Attention is all you need
A Vaswani · 2017
Earlier work this paper cites.
Hierarchical neural story generation
Original
Angela Fan, Mike Lewis, and Yann Dauphin · 2018
Earlier work this paper cites.
The curious case of neural text degeneration
Original
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi · 2019
Earlier work this paper cites.
Calibration of encoder decoder models for neural machine translation
Original
Aviral Kumar and Sunita Sarawagi · 2019
Earlier work this paper cites.
Neural text generation with unlikelihood training
Original
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston · 2019
Earlier work this paper cites.
Fine-tuning language models from human preferences
Original
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving · 2019
Earlier work this paper cites.
Mirostat: A neural text decoding algorithm that directly controls perplexity, 2020
Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, and Lav R. Varshney · 2020
Earlier work this paper cites.

Similar

Trading off diversity and quality in natural language generation
Original
Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan · 2020
Cited alongside, same era.
Training verifiers to solve math word problems
Original
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al · 2021
Cited alongside, same era.
Large language models are zero-shot reasoners
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa · 2022
Cited alongside, same era.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Cited alongside, same era.
Self-consistency improves chain of thought reasoning in language models
Original

Then

Some things are more cringe than others: Preference optimization with the pairwise cringe loss
Original
Jing Xu, Andrew Lee, Sainbayar Sukhbaatar, and Jason Weston · 2023
Later among the works it cites.
Improving code generation by dynamic temperature sampling
Yuqi Zhu, Jia Li, Ge Li, YunFei Zhao, Jia Li, Zhi Jin, and Hong Mei · 2023
Later among the works it cites.
The llama 3 herd of models
Original
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al · 2024
Closest in time.
Dynamic stochastic decoding strategy for open-domain dialogue generation
Original
Yiwei Li, Fei Mi, Yitong Li, Yasheng Wang, Bin Sun, Shaoxiong Feng, and Kan Li · 2024
Closest in time.
Simpo: Simple preference optimization with a reference-free reward

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou · 2022

Cited alongside, same era.

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022

Cited alongside, same era.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023

Cited alongside, same era.

Ultrafeedback: Boosting language models with high-quality feedback

Original

Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun · 2023

Cited alongside, same era.

Closing the curious case of neural text degeneration, 2023

Matthew Finlayson, John Hewitt, Alexander Koller, Swabha Swayamdipta, and Ashish Sabharwal · 2023

Cited alongside, same era.

Understanding the effects of rlhf on llm generalisation and diversity

Original

Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu · 2023

Cited alongside, same era.