WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models

Tydi qa: A benchmark for information-seeking question answering in ty pologically di verse languages

Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki · 2020

Earlier work this paper cites.

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt · 2020

Earlier work this paper cites.

Evaluating large language models trained on code

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al · 2021

Earlier work this paper cites.

Training verifiers to solve math word problems

Original

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al · 2021

Earlier work this paper cites.

Gradient-based adversarial attacks against text transformers, 2021

Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela · 2021

Earlier work this paper cites.

Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan · 2022

Earlier work this paper cites.

Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned, 2022

Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec, Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph, Sam McCandlish, Chris Olah, Jared Kaplan, and Jack Clark · 2022

Earlier work this paper cites.

Toxigen: Controlling language models to generate implied and adversarial toxicity

Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar · 2022

Earlier work this paper cites.

Truthfulqa: Measuring how models mimic human falsehoods

Stephanie Lin, Jacob Hilton, and Owain Evans · 2022

Earlier work this paper cites.

WANLI: Worker and ai collaboration for natural language inference dataset creation

Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi · 2022

Earlier work this paper cites.

Red teaming language models with language models

Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving · 2022

Earlier work this paper cites.

Taxonomy of risks posed by language models

Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, et al · 2022

Earlier work this paper cites.

Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions

Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou · 2023

Earlier work this paper cites.

Executive order on the safe, secure, and trustworthy development and use of artificial intelligence

Joseph R Biden · 2023

Earlier work this paper cites.

Are aligned neural networks adversarially aligned?, 2023

Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, and Ludwig Schmidt · 2023

Earlier work this paper cites.

Explore, establish, exploit: Red teaming language models from scratch, 2023

Stephen Casper, Jason Lin, Joe Kwon, Gatlen Culp, and Dylan Hadfield-Menell · 2023

Earlier work this paper cites.

Jailbreaking black box large language models in twenty queries, 2023

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong · 2023

Earlier work this paper cites.

Code alpaca: An instruction-following llama model for code generation

Sahil Chaudhary · 2023

Earlier work this paper cites.

Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing · 2023

Earlier work this paper cites.

Easylm: A simple and scalable training framework for large language models, 2023

Xinyang Geng · 2023

Earlier work this paper cites.

Time travel in llms: Tracing data contamination in large language models

Shahriar Golchin and Mihai Surdeanu · 2023

Earlier work this paper cites.

An overview of catastrophic ai risks

Original

Dan Hendrycks, Mantas Mazeika, and Thomas Woodside · 2023

Earlier work this paper cites.

Catastrophic jailbreak of open-source llms via exploiting generation, 2023

Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen · 2023

Earlier work this paper cites.

Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023

Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa · 2023

Cited alongside, same era.

Beavertails: Towards improved safety alignment of LLM via a human-preference dataset

Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang · 2023

Cited alongside, same era.

Exploiting programmatic behavior of llms: Dual-use through standard security attacks, 2023

Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto · 2023

Cited alongside, same era.

Open sesame! universal black box jailbreaking of large language models, 2023

Raz Lapid, Ron Langberg, and Moshe Sipper · 2023

Cited alongside, same era.

Alpacaeval: An automatic evaluator of instruction-following models

Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto · 2023

Cited alongside, same era.

Or-bench: An over-refusal benchmark for large language models, 2024

Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh · 2024

Closest in time.

Safe rlhf: Safe reinforcement learning from human feedback

Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang · 2024

Closest in time.

Masterkey: Automated jailbreaking of large language model chatbots

Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu · 2024

Closest in time.

Length-controlled alpacaeval: A simple way to debias automatic evaluators, 2024

Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B. Hashimoto · 2024

Closest in time.

Coercing llms to do and reveal (almost) anything, 2024

Jonas Geiping, Alex Stein, Manli Shu, Khalid Saifullah, Yuxin Wen, and Tom Goldstein · 2024

Closest in time.

Corrective machine unlearning