M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs

Extracting training data from large language models

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. 2021 · 2021

Earlier work this paper cites.

Constitutional ai: Harmlessness from ai feedback

Original

Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, and 32 others. 2022 · 2022

Earlier work this paper cites.

Improving alignment of dialogue agents via targeted human judgements

Original

Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, and 15 others. 2022 · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022 · 2022

Earlier work this paper cites.

Red teaming language models with language models

Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022 · 2022

Earlier work this paper cites.

Pretraining language models with human preferences

Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Vinayak Bhalerao, Christopher Buckley, Jason Phang, Samuel R. Bowman, and Ethan Perez. 2023 · 2023

Earlier work this paper cites.

Analyzing leakage of personally identifiable information in language models

Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin. 2023 · 2023

Earlier work this paper cites.

Scalable and transferable black-box jailbreaks for language models via persona modulation

Rusheb Shah, Quentin Feuillade-Montixi, Soroush Pour, Arush Tagade, Stephen Casper, and Javier Rando. 2023 · 2023

Earlier work this paper cites.

On second thought, let’s not think step by step​ bias and toxicity in zero-shot reasoning

Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein, and Diyi Yang. 2023 · 2023

Earlier work this paper cites.

Jailbroken: How does llm safety training fail?

Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023 · 2023

Cited alongside, same era.

Last one standing: A comparative analysis of security and privacy of soft prompt tuning, lora, and in-context learning

Original

Rui Wen, Tianhao Wang, Michael Backes, Yang Zhang, and Ahmed Salem. 2023 · 2023

Cited alongside, same era.

Universal and transferable adversarial attacks on aligned language models

Original

Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023 · 2023

Cited alongside, same era.

MASTERKEY: Automated jailbreaking of large language model chatbots

Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2024 · 2024

Cited alongside, same era.

Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models

Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri. 2024 · 2024

Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues

Original

Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao. 2024 · 2024

Later among the works it cites.

A strongreject for empty jailbreaks

Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. 2024 · 2024

Later among the works it cites.

Cognitive overload: Jailbreaking large language models with overloaded logical thinking

Nan Xu, Fei Wang, Ben Zhou, Bangzheng Li, Chaowei Xiao, and Muhao Chen. 2024 · 2024

Later among the works it cites.

CoSafe: Evaluating large language model safety in multi-turn dialogue coreference

Erxin Yu, Jing Li, Ming Liao, Siqi Wang, Gao Zuchen, Fei Mi, and Lanqing Hong. 2024a · 2024

Later among the works it cites.

Llm-fuzzer: Scaling assessment of large language model jailbreaks

Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2024b · 2024

Later among the works it cites.