Inverse Constitutional AI: Compressing Preferences into Principles

Judgment under Uncertainty: Heuristics and Biases

Amos Tversky and Daniel Kahneman · 1974

Earlier work this paper cites.

Statistical modeling: The two cultures (with comments and a rejoinder by the author)

Leo Breiman · 2001

Earlier work this paper cites.

Mining Association Rules for Label Ranking

Cláudio Rebelo de Sá, Carlos Soares, Alípio Mário Jorge, Paulo Azevedo, and Joaquim Costa · 2011

Earlier work this paper cites.

Foundations of Rule Learning

Johannes Fürnkranz, Dragan Gamberger, and Nada Lavrač · 2012

Earlier work this paper cites.

Learning to summarize with human feedback

Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano · 2020

Earlier work this paper cites.

Training language models to follow instructions with human feedback, March 2022

Original

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe · 2022

Earlier work this paper cites.

Describing Differences between Text Distributions with Natural Language

Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt · 2022

Earlier work this paper cites.

AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback, August 2023

Original

Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto · 2023

Earlier work this paper cites.

LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion

Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin · 2023

Earlier work this paper cites.

Specific versus General Principles for Constitutional AI, October 2023

Original

Sandipan Kundu, Yuntao Bai, Saurav Kadavath, Amanda Askell, Andrew Callahan, Anna Chen, Anna Goldie, Avital Balwit, Azalia Mirhoseini, Brayden McLean, Catherine Olsson, Cassie Evraets, Eli Tran-Johnson, Esin Durmus, Ethan Perez, Jackson Kernion, Jamie Kerr, Kamal Ndousse, Karina Nguyen, Nelson Elhage, Newton Cheng, Nicholas Schiefer, Nova DasSarma, Oliver Rausch, Robin Larson, Shannon Yang, Shauna Kravec, Timothy Telleen-Lawton, Thomas I. Liao, Tom Henighan, Tristan Hume, Zac Hatfield-Dodds, Sören Mindermann, Nicholas Joseph, Sam McCandlish, and Jared Kaplan · 2023

Earlier work this paper cites.

AlpacaEval: An Automatic Evaluator of Instruction-following Models, May 2024c

Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto · 2023

Earlier work this paper cites.

Calibrating LLM-Based Evaluator, September 2023

Original

Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang · 2023

Earlier work this paper cites.

Biases in Large Language Models: Origins, Inventory, and Discussion

Roberto Navigli, Simone Conia, and Björn Ross · 2023

Earlier work this paper cites.

Direct Preference Optimization: Your Language Model is Secretly a Reward Model, December 2023

Original

Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn · 2023

Earlier work this paper cites.

Towards Understanding Sycophancy in Language Models, October 2023

Original

Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez · 2023

Cited alongside, same era.

Style Over Substance: Evaluation Biases for Large Language Models, November 2023

Original

Minghao Wu and Alham Fikri Aji · 2023

Cited alongside, same era.

A Critical Evaluation of Evaluations for Long-form Question Answering

Fangyuan Xu, Yixiao Song, Mohit Iyyer, and Eunsol Choi · 2023

Cited alongside, same era.

Judging LLM-as-a-judge with MT-Bench and Chatbot Arena, July 2023

Original

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica · 2023

Cited alongside, same era.

The PRISM Alignment Project: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models, April 2024

Original

Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, Bertie Vidgen, and Scott A. Hale · 2024

Closest in time.

Iterative Interactive Inverse Constitutional AI, 2024

Tim Kostolansky and Julian Manyika · 2024

Closest in time.

Inverse Constitutional AI

Timothy H. Kostolansky · 2024

Closest in time.

Does style matter? Disentangling style and substance in Chatbot Arena, 2024b

Tianle Li, Anastasios Angelopoulos, and Wei-Lin Chiang · 2024

Closest in time.

Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs, 2024

Original

Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou · 2024

Closest in time.