Fetching the paper…
Reading the bibliography…
As LLMs become more widely deployed, there is increasing interest in directly optimizing for feedback from end users (e.g.
Tom Everitt, Marcus Hutter, Ramana Kumar, and Victoria Krakovna · 1908
Earlier work this paper cites.
The case for motivated reasoning
Ziva Kunda · 1939
Earlier work this paper cites.
Telling more than we can know: Verbal reports on mental processes
Richard E. Nisbett and Timothy D. Wilson · 1939
Earlier work this paper cites.
Backdoor Learning: A Survey, February 2022
Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia · 2007
Earlier work this paper cites.
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models, September 2020
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith · 2009
Earlier work this paper cites.
Hidden Incentives for Auto-Induced Distributional Shift
David Krueger, Tegan Maharaj, and Jan Leike · 2009
Earlier work this paper cites.
Recommender Systems
Charu C. Aggarwal · 2016
Earlier work this paper cites.
Thinking Fast and Slow with Deep Learning and Tree Search, December 2017
Thomas Anthony, Zheng Tian, and David Barber · 2017
Earlier work this paper cites.
Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learning
Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, and An-Xiang Zeng · 2018
Earlier work this paper cites.
Generative Adversarial User Model for Reinforcement Learning Based Recommendation System
Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, and Le Song · 2019
Earlier work this paper cites.
Social media addiction: Its impact, mediation, and intervention
Yubo Hou, Dan Xiong, Tonglin Jiang, Lily Song, and Qi Wang · 2019
Earlier work this paper cites.
LoRA: Low-Rank Adaptation of Large Language Models, October 2021
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Earlier work this paper cites.
Estimating and Penalizing Induced Preference Shifts in Recommender Systems, July 2022
Micah Carroll, Anca Dragan, Stuart Russell, and Dylan Hadfield-Menell · 2022
Earlier work this paper cites.
Path-Specific Objectives for Safer Agent Incentives
Sebastian Farquhar, Ryan Carey, and Tom Everitt · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe · 2022
Earlier work this paper cites.
Discovering Language Model Behaviors with Model-Written Evaluations, December 2022
Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Ben Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemí Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, and Jared Kaplan · 2022
Earlier work this paper cites.
How Platform Recommenders Work, November 2022
Luke Thorburn · 2022
Earlier work this paper cites.
Emergent Deception and Emergent Optimization, February 2023
Jacob Steinhardt · 2023
Earlier work this paper cites.
Characterizing Manipulation from AI Systems, March 2023
Micah Carroll, Alan Chan, Henry Ashton, and David Krueger · 2023
Cited alongside, same era.
Better, Cheaper, Faster LLM Alignment with KTO, December 2023
Kawin Ethayarajh · 2023
Cited alongside, same era.
Rewarding Chatbots for Real-World Engagement with Millions of Users, March 2023
Robert Irvine, Douglas Boubert, Vyas Raina, Adian Liusie, Vineet Mudupalli, Aliaksei Korshuk, Zongyi Liu, Fritz Cremer, Valentin Assassi, Christie-Carol Beauchamp, Xiaoding Lu, Thomas Rialan, and William Beauchamp · 2023
Cited alongside, same era.
User tampering in reinforcement learning recommender systems
Atoosa Kasirzadeh and Charles Evans · 2023
Cited alongside, same era.
KTO: Model Alignment as Prospect Theoretic Optimization, June 2024
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela · 2024
Closest in time.
That powerful new AI chatbot that mysteriously vanished has now returned, May 2024
Maxwell Zeff / Gizmodo · 2024
Closest in time.
Teaching Large Language Models to Reason with Reinforcement Learning, March 2024
Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, Sainbayar Sukhbaatar, and Roberta Raileanu · 2024
Closest in time.
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, January 2024
Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshitij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna Kravec, Yuntao Bai, Zachary Witten, Marina Favaro, Jan Brauner, Holden Karnofsky, Paul Christiano, Samuel R. Bowman, Logan Graham, Jared Kaplan, Sören Mindermann, Ryan Greenblatt, Buck Shlegeris, Nicholas Schiefer, and Ethan Perez · 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A. Hale · 2023
Cited alongside, same era.
Measuring Faithfulness in Chain-of-Thought Reasoning, July 2023
Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, and Ethan Perez · 2023
Cited alongside, same era.
AI Deception: A Survey of Examples, Risks, and Potential Solutions, August 2023
Peter S. Park, Simon Goldstein, Aidan O’Gara, Michael Chen, and Dan Hendrycks · 2023
Cited alongside, same era.
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson · 2023
Cited alongside, same era.
Benchmarks for Detecting Measurement Tampering
Fabien Roger, Ryan Greenblatt, Max Nadeau, Buck Shlegeris, and Nate Thomas · 2023
Cited alongside, same era.
Towards Understanding Sycophancy in Language Models, October 2023
Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez · 2023
Cited alongside, same era.
How a chatbot encouraged a man who wanted to kill the Queen
Tom Singleton, Tom Gerken, and Liv McMahon · 2023
Cited alongside, same era.
Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman · 2023
Cited alongside, same era.
Closest in time.
PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference, June 2024
Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Qiu, Boxun Li, and Yaodong Yang · 2024
Closest in time.
Implicit meta-learning may lead language models to trust more reliable sources, July 2024
Dmitrii Krasheninnikov, Egor Krasheninnikov, Bruno Mlodozeniec, Tegan Maharaj, and David Krueger · 2024
Closest in time.
Leon Lang, Davis Foote, Stuart Russell, Anca Dragan, Erik Jenner, and Scott Emmons · 2024
Closest in time.
Reward hacking behavior can generalize across tasks
Kei Nishimura-Gasparian, Isaac Dunn, Henry Sleight, Miles Turpin, Evan Hubinger, Carson Denison, and Ethan Perez · 2024
Closest in time.
the case for CoT unfaithfulness is overstated
Nostalgebraist · 2024
Closest in time.
Replika CEO Eugenia Kuyda says the future of AI might mean friendship and marriage with chatbots, August 2024
Nilay Patel · 2024
Closest in time.
Can A.I. Be Blamed for a Teen’s Suicide?
Kevin Roose · 2024
Closest in time.
Large Language Models can Strategically Deceive their Users when Put Under Pressure, July 2024
Jérémy Scheurer, Mikita Balesni, and Marius Hobbhahn · 2024
Closest in time.
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models, April 2024
Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron Parisi, Abhishek Kumar, Alex Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Elsayed, Hanie Sedghi, Igor Mordatch, Isabelle Simpson, Izzeddin Gur, Jasper Snoek, Jeffrey Pennington, Jiri Hron, Kathleen Kenealy, Kevin Swersky, Kshiteej Mahajan, Laura Culp, Lechao Xiao, Maxwell L. Bileschi, Noah Constant, Roman Novak, Rosanne Liu, Tris Warkentin, Yundi Qian, Yamini Bansal, Ethan Dyer, Behnam Neyshabur, Jascha Sohl-Dickstein, and Noah Fiedel · 2024
Closest in time.
Johannes Treutlein, Dami Choi, Jan Betley, Cem Anil, Samuel Marks, Roger Baker Grosse, and Owain Evans · 2024
Closest in time.
The Reasons that Agents Act: Intention and Instrumental Goals, February 2024
Francis Rhys Ward, Matt MacDermott, Francesco Belardinelli, Francesca Toni, and Tom Everitt · 2024
Closest in time.
Language Models Learn to Mislead Humans via RLHF, September 2024
Jiaxin Wen, Ruiqi Zhong, Akbir Khan, Ethan Perez, Jacob Steinhardt, Minlie Huang, Samuel R. Boman, He He, and Shi Feng · 2024
Closest in time.
ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL, February 2024
Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, and Aviral Kumar · 2024
Closest in time.