SeqAR: Jailbreak LLMs with Sequential Auto-Generated Characters

Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned

Original

Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. 2022 · 2022

Earlier work this paper cites.

Introducing chatgpt

OpenAI. 2022 · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Earlier work this paper cites.

Red teaming language models with language models

Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022 · 2022

Earlier work this paper cites.

Are aligned neural networks adversarially aligned?

Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Pang Wei W Koh, Daphne Ippolito, Florian Tramer, and Ludwig Schmidt. 2023 · 2023

Earlier work this paper cites.

Jailbreaking black box large language models in twenty queries

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2023 · 2023

Earlier work this paper cites.

Chat gpt "dan" (and other "jailbreaks")

DAN. 2023 · 2023

Earlier work this paper cites.

Analyzing the inherent response tendency of llms: Real-world instructions-driven jailbreak

Original

Yanrui Du, Sendong Zhao, Ming Ma, Yuhan Chen, and Bing Qin. 2023 · 2023

Earlier work this paper cites.

DeBERTav3: Improving deBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing

Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023 · 2023

Earlier work this paper cites.

Baseline Defenses for Adversarial Attacks Against Aligned Language Models

Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2023 · 2023

Earlier work this paper cites.

Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study

Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. 2023 · 2023

Earlier work this paper cites.

Gpt-4 technical report

Original

OpenAI. 2023 · 2023

Earlier work this paper cites.

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023 · 2023

Earlier work this paper cites.

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks

Alexander Robey, Eric Wong, Hamed Hassani, and George J. Pappas Pappas. 2023 · 2023

Earlier work this paper cites.

Large Language Models Can Be Easily Distracted by Irrelevant Context

Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. 2023 · 2023

Cited alongside, same era.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023

Cited alongside, same era.

Defending chatgpt against jailbreak attack via self-reminders

Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023 · 2023

Cited alongside, same era.

Low-resource languages jailbreak GPT-4

Zheng Xin Yong, Cristina Menghini, and Stephen Bach. 2023 · 2023

Cited alongside, same era.

GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts

Original

Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2023 · 2023

Deepinception: Hypnotize large language model to be jailbreaker

Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. 2024 · 2024

Closest in time.

AmpleGCG: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed LLMs

Zeyi Liao and Huan Sun. 2024 · 2024

Closest in time.

Harmbench: a standardized evaluation framework for automated red teaming and robust refusal

Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. 2024 · 2024

Closest in time.

Tree of attacks: Jailbreaking black-box llms automatically

Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2024 · 2024

Closest in time.

Hello gpt-4o

OpenAI. 2024 · 2024

Closest in time.

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context