ComPO: Community Preferences for Language Model Personalization

Fine-tuning language models from human preferences

Original

Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019 · 1909

Earlier work this paper cites.

Rank analysis of incomplete block designs: I. the method of paired comparisons

Ralph Allan Bradley and Milton E Terry. 1952 · 1952

Earlier work this paper cites.

Probabilistic matrix factorization

Andriy Mnih and Russ R Salakhutdinov. 2007 · 2007

Earlier work this paper cites.

Factorization machines

Steffen Rendle. 2010 · 2010

Earlier work this paper cites.

Science, askscience, and badscience: On the coexistence of highly related communities

Jack Hessel, Chenhao Tan, and Lillian Lee. 2016 · 2016

Earlier work this paper cites.

Bag of tricks for efficient text classification

Original

Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016 · 2016

Earlier work this paper cites.

Deep reinforcement learning from human preferences

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017 · 2017

Earlier work this paper cites.

Deep learning based recommender system: A survey and new perspectives

Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019 · 2019

Earlier work this paper cites.

The pushshift reddit dataset

Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020 · 2020

Earlier work this paper cites.

The curious case of neural text degeneration

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020 · 2020

Earlier work this paper cites.

Learning to summarize with human feedback

Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020 · 2020

Earlier work this paper cites.

Advances in collaborative filtering

Yehuda Koren, Steffen Rendle, and Robert Bell. 2021 · 2021

Earlier work this paper cites.

Useridentifier: implicit user representations for simple and effective personalized sentiment analysis

Original

Fatemehsadat Mireshghallah, Vaishnavi Shrivastava, Milad Shokouhi, Taylor Berg-Kirkpatrick, Robert Sim, and Dimitrios Dimitriadis. 2021 · 2021

Earlier work this paper cites.

Webgpt: Browser-assisted question-answering with human feedback

Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2021 · 2021

Earlier work this paper cites.

Process for adapting language models to society (palms) with values-targeted datasets

Irene Solaiman and Christy Dennison. 2021 · 2021

Earlier work this paper cites.

Training a helpful and harmless assistant with reinforcement learning from human feedback

Original

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022 · 2022

Cited alongside, same era.

Fine-tuning language models to find agreement among humans with diverse preferences

Michiel Bakker, Martin Chadwick, Hannah Sheahan, Michael Tessler, Lucy Campbell-Gillingham, Jan Balaguer, Nat McAleese, Amelia Glaese, John Aslanides, Matt Botvinick, et al. 2022 · 2022

Cited alongside, same era.

Implicit preferences

Tom Cunningham and Jonathan De Quidt. 2022 · 2022

Cited alongside, same era.

Dealing with disagreements: Looking beyond the majority vote in subjective annotations

Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022 · 2022

Cited alongside, same era.

Understanding dataset difficulty with 𝒱 \mathcal{V} -usable information

Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. 2022 · 2022

Cited alongside, same era.

The past, present and better future of feedback learning in large language models for subjective human preferences and values

Hannah Kirk, Andrew Bean, Bertie Vidgen, Paul Rottger, and Scott Hale. 2023 · 2023

Later among the works it cites.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023

Later among the works it cites.

Zephyr: Direct distillation of lm alignment

Original

Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. 2023 · 2023

Later among the works it cites.

Chatgpt is more likely to be perceived as male than female

Original

Jared Wong and Jin Kim. 2023 · 2023

Later among the works it cites.