The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Models

Deep reinforcement learning from human preferences

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017 · 2017

Earlier work this paper cites.

Scalable agent alignment via reward modeling: a research direction

Original

Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. 2018 · 2018

Earlier work this paper cites.

Aligning AI with shared human values

Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021 · 2021

Earlier work this paper cites.

Improving alignment of dialogue agents via targeted human judgements

Original

Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, et al. 2022 · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Earlier work this paper cites.

Jailbreaking black box large language models in twenty queries

Original

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023 · 2023

Earlier work this paper cites.

A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily

Original

Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2023 · 2023

Earlier work this paper cites.

Pretraining language models with human preferences

Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Vinayak Bhalerao, Christopher Buckley, Jason Phang, Samuel R Bowman, and Ethan Perez. 2023 · 2023

Earlier work this paper cites.

Lora fine-tuning efficiently undoes safety training in llama 2-chat 70b

Original

Simon Lermen, Charlie Rogers-Smith, and Jeffrey Ladish. 2023 · 2023

Earlier work this paper cites.

Autodan: Generating stealthy jailbreak prompts on aligned large language models

Original

Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023 · 2023

Earlier work this paper cites.

Tree of attacks: Jailbreaking black-box llms automatically

Original

Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2023 · 2023

Earlier work this paper cites.

Language model alignment with elastic reset

Michael Noukhovitch, Samuel Lavoie, Florian Strub, and Aaron C. Courville. 2023 · 2023

Cited alongside, same era.

Exploiting novel gpt-4 apis

Original

Kellin Pelrine, Mohammad Taufeeque, Michał Zając, Euan McLean, and Adam Gleave. 2023 · 2023

Cited alongside, same era.

Fine-tuning aligned language models compromises safety, even when users do not intend to!

Original

Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023 · 2023

Cited alongside, same era.

" do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models

Original

Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023 · 2023

Cited alongside, same era.

Sight beyond text: Multi-modal training enhances llms in truthfulness and ethics

Claude 3.5 sonnet

Anthropic. 2024 · 2024

Closest in time.

Chatglm: A family of large language models from glm-130b to glm-4 all tools

Original

Team GLM, :, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao, Shuxun Yang, Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang, Yueyan Li, Yushi Bai, Yuxiao Dong, Zehan Qi, Zhaoyu Wang, Zhen Yang, Zhengxiao Du, Zhenyu Hou, and Zihan Wang. 2024 · 2024

Closest in time.

Mixtral of experts

Original

Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024 · 2024

Closest in time.

A cross-language investigation into jailbreak attacks in large language models