Built on

Measuring massive multitask language understanding
Original
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020 · 2009
Earlier work this paper cites.
Stackelberg games for adversarial prediction problems
Michael Brückner and Tobias Scheffer. 2011 · 2011
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Original
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014 · 2014
Earlier work this paper cites.
Towards deep learning models resistant to adversarial attacks
Original
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017 · 2017
Earlier work this paper cites.
Billion-scale similarity search with gpus
Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019 · 2019
Earlier work this paper cites.
Gpt-3: Its nature, scope, limits, and consequences
Luciano Floridi and Massimo Chiriatti. 2020 · 2020
Earlier work this paper cites.
Finetuned language models are zero-shot learners
Original
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021 · 2021
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022
Earlier work this paper cites.
Detecting language model attacks with perplexity
Original
Gabriel Alon and Michael Kamfonas. 2023 · 2023
Earlier work this paper cites.
Red-teaming large language models using chain of utterances for safety-alignment
Original
Rishabh Bhardwaj and Soujanya Poria. 2023 · 2023
Earlier work this paper cites.
Jailbreaking black box large language models in twenty queries
Original
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023 · 2023
Earlier work this paper cites.
Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023 · 2023
Earlier work this paper cites.
Multilingual jailbreak challenges in large language models
Original
Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2023 · 2023
Earlier work this paper cites.
Improving language model negotiation with self-play and in-context learning from ai feedback
Original
Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. 2023 · 2023
Earlier work this paper cites.
Mart: Improving llm safety with multi-round automatic red-teaming
Original
Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. 2023 · 2023
Earlier work this paper cites.
Llm self defense: By self examination, llms know they are being tricked
Original
Alec Helbling, Mansi Phute, Matthew Hull, and Duen Horng Chau. 2023 · 2023
Earlier work this paper cites.

Similar

Catastrophic jailbreak of open-source llms via exploiting generation
Original
Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2023 · 2023
Cited alongside, same era.
Llama guard: Llm-based input-output safeguard for human-ai conversations
Original
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023 · 2023
Cited alongside, same era.
Baseline defenses for adversarial attacks against aligned language models
Original
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2023 · 2023
Cited alongside, same era.
Mistral 7b
Original
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023
Cited alongside, same era.

Then

The art of defending: A systematic evaluation and analysis of llm defense strategies on safety and over-defensiveness
Original
Neeraj Varshney, Pavel Dolin, Agastya Seth, and Chitta Baral. 2023 · 2023
Later among the works it cites.
Self-guard: Empower the llm to safeguard itself
Original
Zezhong Wang, Fangkai Yang, Lu Wang, Pu Zhao, Hongru Wang, Liang Chen, Qingwei Lin, and Kam-Fai Wong. 2023 · 2023
Later among the works it cites.
The rise and potential of large language model based agents: A survey
Original
Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023 · 2023
Later among the works it cites.
Defending chatgpt against jailbreak attack via self-reminders
Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023 · 2023
Later among the works it cites.
Retroformer: Retrospective large language agents with policy gradient optimization

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Defending Jailbreak Prompts via In-Context Adversarial Game

Zhou, Yujun, Han, Yufei, Zhuang, Haomin et al.

Understand

Large Language Models (LLMs) demonstrate remarkable capabilities across diverse applications.

However, concerns regarding their security, particularly the vulnerability to jailbreak attacks, persist.
Drawing inspiration from adversarial training in deep learning and LLM agent learning processes, we introduce the In-Context Adversarial Game (ICAG) for defending against jailbreaks without the need for fine-tuning.
ICAG leverages agent learning to conduct an adversarial game, aiming to dynamically extend knowledge to defend against jailbreaks.

Built on

Measuring massive multitask language understanding
Original
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020 · 2009
Earlier work this paper cites.
Stackelberg games for adversarial prediction problems
Michael Brückner and Tobias Scheffer. 2011 · 2011
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Original
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014 · 2014
Earlier work this paper cites.
Towards deep learning models resistant to adversarial attacks
Original
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017 · 2017
Earlier work this paper cites.
Billion-scale similarity search with gpus
Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019 · 2019
Earlier work this paper cites.
Gpt-3: Its nature, scope, limits, and consequences
Luciano Floridi and Massimo Chiriatti. 2020 · 2020
Earlier work this paper cites.
Finetuned language models are zero-shot learners
Original
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021 · 2021
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022
Earlier work this paper cites.
Detecting language model attacks with perplexity
Original
Gabriel Alon and Michael Kamfonas. 2023 · 2023
Earlier work this paper cites.
Red-teaming large language models using chain of utterances for safety-alignment
Original
Rishabh Bhardwaj and Soujanya Poria. 2023 · 2023
Earlier work this paper cites.
Jailbreaking black box large language models in twenty queries
Original
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023 · 2023
Earlier work this paper cites.
Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023 · 2023
Earlier work this paper cites.
Multilingual jailbreak challenges in large language models
Original
Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2023 · 2023
Earlier work this paper cites.
Improving language model negotiation with self-play and in-context learning from ai feedback
Original
Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. 2023 · 2023
Earlier work this paper cites.
Mart: Improving llm safety with multi-round automatic red-teaming
Original
Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. 2023 · 2023
Earlier work this paper cites.
Llm self defense: By self examination, llms know they are being tricked
Original
Alec Helbling, Mansi Phute, Matthew Hull, and Duen Horng Chau. 2023 · 2023
Earlier work this paper cites.

Similar

Catastrophic jailbreak of open-source llms via exploiting generation
Original
Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2023 · 2023
Cited alongside, same era.
Llama guard: Llm-based input-output safeguard for human-ai conversations
Original
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023 · 2023
Cited alongside, same era.
Baseline defenses for adversarial attacks against aligned language models
Original
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2023 · 2023
Cited alongside, same era.
Mistral 7b
Original
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023
Cited alongside, same era.

Then

The art of defending: A systematic evaluation and analysis of llm defense strategies on safety and over-defensiveness
Original
Neeraj Varshney, Pavel Dolin, Agastya Seth, and Chitta Baral. 2023 · 2023
Later among the works it cites.
Self-guard: Empower the llm to safeguard itself
Original
Zezhong Wang, Fangkai Yang, Lu Wang, Pu Zhao, Hongru Wang, Liang Chen, Qingwei Lin, and Kam-Fai Wong. 2023 · 2023
Later among the works it cites.
The rise and potential of large language model based agents: A survey
Original
Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023 · 2023
Later among the works it cites.
Defending chatgpt against jailbreak attack via self-reminders
Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023 · 2023
Later among the works it cites.
Retroformer: Retrospective large language agents with policy gradient optimization

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Certifying llm safety against adversarial prompting

Original

Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Soheil Feizi, and Hima Lakkaraju. 2023 · 2023

Cited alongside, same era.

Multi-step jailbreaking privacy attacks on chatgpt

Original

Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. 2023 · 2023

Cited alongside, same era.

Autodan: Generating stealthy jailbreak prompts on aligned large language models

Original

Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023 · 2023

Cited alongside, same era.

Red teaming game: A game-theoretic framework for red teaming language models

Original

Chengdong Ma, Ziran Yang, Minquan Gao, Hai Ci, Jun Gao, Xuehai Pan, and Yaodong Yang. 2023 · 2023

Cited alongside, same era.

Gpt-4 technical report

Original

OpenAI. 2023 · 2023

Cited alongside, same era.

Smoothllm: Defending large language models against jailbreaking attacks

Original

Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. 2023 · 2023

Cited alongside, same era.

Xstest: A test suite for identifying exaggerated safety behaviours in large language models

Original

Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2023 · 2023

Cited alongside, same era.

Scalable and transferable black-box jailbreaks for language models via persona modulation

Original

Rusheb Shah, Soroush Pour, Arush Tagade, Stephen Casper, Javier Rando, et al. 2023 · 2023

Cited alongside, same era.

Original

Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh Murthy, Zeyuan Chen, Jianguo Zhang, Devansh Arpit, et al. 2023 · 2023

Later among the works it cites.

Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher

Original

Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2023 · 2023

Later among the works it cites.

Defending large language models against jailbreaking attacks through goal prioritization

Original

Zhexin Zhang, Junxiao Yang, Pei Ke, and Minlie Huang. 2023 · 2023

Later among the works it cites.

Expel: Llm agents are experiential learners

Original

Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2023 · 2023