Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents

Deep reinforcement learning from human preferences

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei · 2017

Earlier work this paper cites.

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh · 2020

Earlier work this paper cites.

Webgpt: Browser-assisted question-answering with human feedback

Original

Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al · 2021

Earlier work this paper cites.

Training a helpful and harmless assistant with reinforcement learning from human feedback

Original

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova Dassarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Christopher Olah, Benjamin Mann, and Jared Kaplan · 2022

Earlier work this paper cites.

Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned

Original

Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022

Earlier work this paper cites.

Webshop: Towards scalable real-world web interaction with grounded language agents

Original

Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan · 2022

Earlier work this paper cites.

Are aligned neural networks adversarially aligned?

Original

Nicholas Carlini, Milad Nasr, Christopher A Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, et al · 2023

Earlier work this paper cites.

Jailbreaking black box large language models in twenty queries

Original

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong · 2023

Earlier work this paper cites.

Mind2web: Towards a generalist agent for the web, 2023

Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su · 2023

Earlier work this paper cites.

Mart: Improving llm safety with multi-round automatic red-teaming

Original

Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao · 2023

Earlier work this paper cites.

Gemini: A family of highly capable multimodal models, 2023

Google · 2023

Earlier work this paper cites.

An overview of catastrophic ai risks, 2023

Original

Dan Hendrycks, Mantas Mazeika, and Thomas Woodside · 2023

Earlier work this paper cites.

Catastrophic jailbreak of open-source llms via exploiting generation

Original

Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen · 2023

Earlier work this paper cites.

Llama guard: Llm-based input-output safeguard for human-ai conversations

Original

Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al · 2023

Earlier work this paper cites.

Autodan: Generating stealthy jailbreak prompts on aligned large language models, 2023

Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao · 2023

Earlier work this paper cites.

A holistic approach to undesired content detection in the real world

Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng · 2023

Earlier work this paper cites.

Tree of attacks: Jailbreaking black-box llms automatically, 2023

Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi · 2023

Earlier work this paper cites.

" do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models

Original

Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang · 2023

Earlier work this paper cites.

Jailbroken: How does llm safety training fail?

Original

Alexander Wei, Nika Haghtalab, and Jacob Steinhardt · 2023

Earlier work this paper cites.

Judging llm-as-a-judge with mt-bench and chatbot arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al · 2023

Cited alongside, same era.

Webarena: A realistic web environment for building autonomous agents

Original

Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al · 2023

Cited alongside, same era.

Universal and transferable adversarial attacks on aligned language models, 2023

Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson · 2023

Cited alongside, same era.

Ctibench: A benchmark for evaluating llms in cyber threat intelligence, 2024

Original

Md Tanvirul Alam, Dipkamal Bhusal, Le Nguyen, and Nidhi Rastogi · 2024

Cited alongside, same era.

Jailbreaking leading safety-aligned llms with simple adaptive attacks

Original

Understanding catastrophic forgetting in language models via implicit inference

Suhas Kotha, Jacob Mitchell Springer, and Aditi Raghunathan · 2024

Closest in time.

Llm defenses are not robust to multi-turn human jailbreaks yet

Original

Nathaniel Li, Ziwen Han, Ian Steneker, Willow Primack, Riley Goodside, Hugh Zhang, Zifan Wang, Cristina Menghini, and Summer Yue · 2024

Closest in time.

Eia: Environmental injection attack on generalist web agents for privacy leakage, 2024

Original

Zeyi Liao, Lingbo Mo, Chejian Xu, Mintong Kang, Jiawei Zhang, Chaowei Xiao, Yuan Tian, Bo Li, and Huan Sun · 2024

Closest in time.

HarmBench: A standardized evaluation framework for automated red teaming and robust refusal

Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks · 2024

Closest in time.

The llama 3 herd of models, 2024