BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignment

Extensions of lipshitz mapping into hilbert space

William B Johnson. 1984 · 1984

Earlier work this paper cites.

Adam: A method for stochastic optimization

Original

Diederik P Kingma. 2014 · 2014

Earlier work this paper cites.

Deep reinforcement learning from human preferences

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017 · 2017

Earlier work this paper cites.

Proximal policy optimization algorithms

Original

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017 · 2017

Earlier work this paper cites.

Estimating training data influence by tracing gradient descent

Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. 2020 · 2020

Earlier work this paper cites.

Lora: Low-rank adaptation of large language models

Original

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

Earlier work this paper cites.

Training a helpful and harmless assistant with reinforcement learning from human feedback

Original

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022 · 2022

Earlier work this paper cites.

Calibrating sequence likelihood improves conditional language generation

Yao Zhao, Mikhail Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. 2022 · 2022

Earlier work this paper cites.

Ultrafeedback: Boosting language models with high-quality feedback

Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023 · 2023

Earlier work this paper cites.

Safe rlhf: Safe reinforcement learning from human feedback

Original

Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2023 · 2023

Earlier work this paper cites.

Human-like summarization evaluation with chatgpt

Original

Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, and Xiaojun Wan. 2023 · 2023

Earlier work this paper cites.

Understanding in-context learning via supportive pretraining data

Xiaochuang Han, Daniel Simig, Todor Mihaylov, Yulia Tsvetkov, Asli Celikyilmaz, and Tianlu Wang. 2023 · 2023

Earlier work this paper cites.

Alpacaeval: An automatic evaluator of instruction-following models

Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023 · 2023

Cited alongside, same era.

What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning

Original

Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2023 · 2023

Cited alongside, same era.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023

Cited alongside, same era.

Instructiongpt-4: A 200-instruction paradigm for fine-tuning minigpt-4

Original

Lai Wei, Zihao Jiang, Weiran Huang, and Lichao Sun. 2023 · 2023

Cited alongside, same era.

A survey on data selection for language models

Simpo: Simple preference optimization with a reference-free reward

Original

Yu Meng, Mengzhou Xia, and Danqi Chen. 2024 · 2024

Closest in time.

Filtered direct preference optimization

Original

Tetsuro Morimura, Mitsuki Sakamoto, Yuu Jinnai, Kenshi Abe, and Kaito Air. 2024 · 2024

Closest in time.

Text clustering with llm embeddings

Original

Alina Petukhova, Joao P Matos-Carvalho, and Nuno Fachada. 2024 · 2024

Closest in time.

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024 · 2024

Closest in time.

Insights into alignment: Evaluating dpo and its variants across multiple tasks

Original