Noise Contrastive Alignment of Language Models with Explicit Rewards

Rank analysis of incomplete block designs: I. the method of paired comparisons

Ralph Allan Bradley and Milton E Terry · 1952

Earlier work this paper cites.

Reinforcement learning by reward-weighted regression for operational space control

Jan Peters and Stefan Schaal · 2007

Earlier work this paper cites.

Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics

Michael U Gutmann and Aapo Hyvärinen · 2012

Earlier work this paper cites.

Distributed representations of words and phrases and their compositionality

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean · 2013

Earlier work this paper cites.

Deep reinforcement learning from human preferences

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei · 2017

Earlier work this paper cites.

Proximal policy optimization algorithms

Original

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov · 2017

Earlier work this paper cites.

Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency

Original

Zhuang Ma and Michael Collins · 2018

Earlier work this paper cites.

Representation learning with contrastive predictive coding

Original

Aaron van den Oord, Yazhe Li, and Oriol Vinyals · 2018

Earlier work this paper cites.

Advantage-weighted regression: Simple and scalable off-policy reinforcement learning

Original

Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine · 2019

Earlier work this paper cites.

A simple framework for contrastive learning of visual representations

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton · 2020

Earlier work this paper cites.

Momentum contrast for unsupervised visual representation learning

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick · 2020

Earlier work this paper cites.

Curl: Contrastive unsupervised representations for reinforcement learning

Michael Laskin, Aravind Srinivas, and Pieter Abbeel · 2020

Earlier work this paper cites.

A diverse corpus for evaluating and developing English math word problem solvers

Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su · 2020

Earlier work this paper cites.

TRL: Transformer reinforcement learning, 2020

Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, and Shengyi Huang · 2020

Earlier work this paper cites.

Evaluating large language models trained on code, 2021

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba · 2021

Earlier work this paper cites.

InfoXLM: An information-theoretic framework for cross-lingual language model pre-training

Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou · 2021

Earlier work this paper cites.

Are NLP models really able to solve simple math word problems?

Arkil Patel, Satwik Bhattamishra, and Navin Goyal · 2021

Cited alongside, same era.

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021

Cited alongside, same era.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022

Cited alongside, same era.

Chatgpt: Optimizing language models for dialogue

John Schulman, Barret Zoph, Christina Kim, Jacob Hilton, Jacob Menick, Jiayi Weng, Juan Felipe Ceron Uribe, Liam Fedus, Luke Metz, Michael Pokorny, et al · 2022

Cited alongside, same era.

Challenging big-bench tasks and whether chain-of-thought can solve them

Original

Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei · 2022

RRHF: Rank responses to align language models with human feedback

Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang · 2023

Later among the works it cites.

SLIC-HF: Sequence likelihood calibration with human feedback

Original

Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu · 2023

Later among the works it cites.

Judging llm-as-a-judge with mt-bench and chatbot arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al · 2023

Later among the works it cites.

A general theoretical paradigm to understand learning from human preferences

Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello · 2024

Closest in time.

Ultrafeedback: Boosting language models with scaled ai feedback

Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, et al · 2024

Closest in time.