SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling

Language models are few-shot learners

Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020 · 1901

Earlier work this paper cites.

Fine-tuning language models from human preferences

Original

Ziegler, D. M.; Stiennon, N.; Wu, J.; Brown, T. B.; Radford, A.; Amodei, D.; Christiano, P.; and Irving, G. 2019 · 1909

Earlier work this paper cites.

Advantage-weighted regression: Simple and scalable off-policy reinforcement learning

Original

Peng, X. B.; Kumar, A.; Zhang, G.; and Levine, S. 2019 · 1910

Earlier work this paper cites.

Benchmarking safe exploration in deep reinforcement learning

Original

Ray, A.; Achiam, J.; and Amodei, D. 2019 · 1910

Earlier work this paper cites.

Rank analysis of incomplete block designs: I. The method of paired comparisons

Bradley, R. A.; and Terry, M. E. 1952 · 1952

Earlier work this paper cites.

Proximal policy optimization algorithms

Original

Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017 · 2017

Earlier work this paper cites.

Learning to summarize with human feedback

Stiennon, N.; Ouyang, L.; Wu, J.; Ziegler, D.; Lowe, R.; Voss, C.; Radford, A.; Amodei, D.; and Christiano, P. F. 2020 · 2020

Earlier work this paper cites.

Lora: Low-rank adaptation of large language models

Original

Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2021 · 2021

Earlier work this paper cites.

Multitask prompted training enables zero-shot task generalization

Original

Sanh, V.; Webson, A.; Raffel, C.; Bach, S. H.; Sutawika, L.; Alyafeai, Z.; Chaffin, A.; Stiegler, A.; Scao, T. L.; Raja, A.; et al. 2021 · 2021

Earlier work this paper cites.

Training a helpful and harmless assistant with reinforcement learning from human feedback

Original

Bai, Y.; Jones, A.; Ndousse, K.; Askell, A.; Chen, A.; DasSarma, N.; Drain, D.; Fort, S.; Ganguli, D.; Henighan, T.; et al. 2022 · 2022

Earlier work this paper cites.

Scaling instruction-finetuned language models

Original

Chung, H. W.; Hou, L.; Longpre, S.; Zoph, B.; Tay, Y.; Fedus, W.; Li, Y.; Wang, X.; Dehghani, M.; Brahma, S.; et al. 2022 · 2022

Earlier work this paper cites.

Codegen: An open large language model for code with multi-turn program synthesis

Original

Nijkamp, E.; Pang, B.; Hayashi, H.; Tu, L.; Wang, H.; Zhou, Y.; Savarese, S.; and Xiong, C. 2022 · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. 2022 · 2022

Earlier work this paper cites.

The AI Economist: Taxation policy design via two-level deep multiagent reinforcement learning

Zheng, S.; Trott, A.; Srinivasa, S.; Parkes, D. C.; and Socher, R. 2022 · 2022

Cited alongside, same era.

Model Card and Evaluations for Claude Models

Anthropic. 2023 · 2023

Cited alongside, same era.

Safe rlhf: Safe reinforcement learning from human feedback

Original

Dai, J.; Pan, X.; Sun, R.; Ji, J.; Xu, X.; Liu, M.; Wang, Y.; and Yang, Y. 2023 · 2023

Cited alongside, same era.

Personalized soups: Personalized large language model alignment via post-hoc parameter merging

Original

Jang, J.; Kim, S.; Lin, B. Y.; Wang, Y.; Hessel, J.; Zettlemoyer, L.; Hajishirzi, H.; Choi, Y.; and Ammanabrolu, P. 2023 · 2023

Cited alongside, same era.

Beavertails: Towards improved safety alignment of llm via a human-preference dataset

Original

Ji, J.; Liu, M.; Dai, J.; Pan, X.; Zhang, C.; Bian, C.; Sun, R.; Wang, Y.; and Yang, Y. 2023 · 2023

Stanford Alpaca: An Instruction-following LLaMA model

Taori, R.; Gulrajani, I.; Zhang, T.; Dubois, Y.; Li, X.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023 · 2023

Later among the works it cites.

On Diverse Preferences for Large Language Model Alignment

Original

Zeng, D.; Dai, Y.; Cheng, P.; Hu, T.; Chen, W.; Du, N.; and Xu, Z. 2023 · 2023

Later among the works it cites.

Building cooperative embodied agents modularly with large language models

Original

Zhang, H.; Du, W.; Shan, J.; Zhou, Q.; Du, Y.; Tenenbaum, J. B.; Shu, T.; and Gan, C. 2023 · 2023

Later among the works it cites.

Judging LLM-as-a-judge with MT-Bench and Chatbot Arena

Original

Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E.; et al. 2023 · 2023

Later among the works it cites.

Nemotron-4 340B Technical Report