Language Models Resist Alignment: Evidence From Data Compression

The psychology of language

George Kingsley Zipf. 1946 · 1946

Earlier work this paper cites.

A mathematical theory of communication

Claude Elwood Shannon. 1948 · 1948

Earlier work this paper cites.

A method for the construction of minimum-redundancy codes

David A Huffman. 1952 · 1952

Earlier work this paper cites.

Cooperation under the security dilemma

Robert Jervis. 1978 · 1978

Earlier work this paper cites.

Arithmetic coding for data compression

Ian H Witten, Radford M Neal, and John G Cleary. 1987 · 1987

Earlier work this paper cites.

Scaling laws for neural language models

Original

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020 · 2001

Earlier work this paper cites.

Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probability

Marcus Hutter. 2005 · 2005

Earlier work this paper cites.

Power laws, pareto distributions and zipf’s law

Mark EJ Newman. 2005 · 2005

Earlier work this paper cites.

Is this paper dangerous? balancing secrecy and openness in counterterrorism

Jacob N Shapiro and David A Siegel. 2010 · 2010

Earlier work this paper cites.

Learning word vectors for sentiment analysis

Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011 · 2011

Earlier work this paper cites.

Lectures de potentia restitutiva, or of spring explaining the power of springing bodies

Robert Hooke. 2016 · 2016

Earlier work this paper cites.

A survey of transfer learning

Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. 2016 · 2016

Earlier work this paper cites.

A comprehensive survey on transfer learning

Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. 2020 · 2020

Earlier work this paper cites.

A general language assistant as a laboratory for alignment

Original

Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. 2021 · 2021

Earlier work this paper cites.

How does the offense-defense balance scale?

Ben Garfinkel and Allan Dafoe. 2021 · 2021

Earlier work this paper cites.

Artificial intelligence act

Tambiama Madiega. 2021 · 2021

Earlier work this paper cites.

Is sgd a bayesian sampler? well, almost

Chris Mingard, Guillermo Valle-Pérez, Joar Skalse, and Ard A Louis. 2021 · 2021

Earlier work this paper cites.

Truthfulqa: Measuring how models mimic human falsehoods

Stephanie Lin, Jacob Hilton, and Owain Evans. 2022 · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Earlier work this paper cites.

Dual use of artificial-intelligence-powered drug discovery

Fabio Urbina, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Earlier work this paper cites.

Qwen technical report

Original

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023 · 2023

Earlier work this paper cites.

Open problems and fundamental limitations of reinforcement learning from human feedback

Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Tong Wang, Samuel Marks, Charbel-Raphael Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman Anwar, Anand Siththaranjan, Max Nadeau, Eric J Michaud, Jacob Pfau, Dmitrii Krasheninnikov, Xin Chen, Lauro Langosco, Peter Hase, Erdem Biyik, Anca Dragan, David Krueger, Dorsa Sadigh, and Dylan Hadfield-Menell. 2023 · 2023

Earlier work this paper cites.

Language modeling is compression

Original

Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, et al. 2023 · 2023

Earlier work this paper cites.

Raft: Reward ranked finetuning for generative foundation model alignment

Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, SHUM KaShun, and Tong Zhang. 2023 · 2023

Earlier work this paper cites.

Generative language models and automated influence operations: Emerging threats and potential mitigations

Original

Josh A Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. 2023 · 2023

Earlier work this paper cites.

Reinforced self-training (rest) for language modeling

Original

Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. 2023 · 2023

Cited alongside, same era.

More than a feeling: Accuracy and application of sentiment analysis

Jochen Hartmann, Mark Heitmann, Christian Siebert, and Christina Schamp. 2023 · 2023

Cited alongside, same era.

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

Original

Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P Dick, Hidenori Tanaka, Edward Grefenstette, Tim Rocktäschel, and David Scott Krueger. 2023 · 2023

Cited alongside, same era.

Ai alignment: A comprehensive survey

Original

Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, et al. 2023 · 2023

Cited alongside, same era.

Consensus statement on red lines in artificial intelligence

IDAIS-Beijing. 2024 · 2024

Closest in time.

A literature survey on open source large language models

Sanjay Kukreja, Tarun Kumar, Amit Purohit, Abhijit Dasgupta, and Debashis Guha. 2024 · 2024

Closest in time.

When your ais deceive you: Challenges of partial observability in reinforcement learning from human feedback

Leon Lang, Davis Foote, Stuart J Russell, Anca Dragan, Erik Jenner, and Scott Emmons. 2024 · 2024

Closest in time.

Rrm: Robust reward model training mitigates reward hacking

Original

Tianqi Liu, Wei Xiong, Jie Ren, Lichang Chen, Junru Wu, Rishabh Joshi, Yang Gao, Jiaming Shen, Zhen Qin, Tianhe Yu, et al. 2024 · 2024

Closest in time.

Simpo: Simple preference optimization with a reference-free reward

Yu Meng, Mengzhou Xia, and Danqi Chen. 2024 · 2024

Closest in time.