JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models

F. K. McDuff and S. D. Turner, “Jailbreak: oncogene-induced senescence and its evasion,” Cellular signalling

2011

Earlier work this paper cites.

N. Papernot, P. McDaniel, and I. Goodfellow, “Transferability in machine learning: from phenomena to black-box attacks using adversarial samples,” arXiv preprint arXiv:1605.07277

Original

2016

Earlier work this paper cites.

P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems

2017

Earlier work this paper cites.

Y. Liu, X. Chen, C. Liu, and D. Song, “Delving into transferable adversarial examples and black-box attacks,” 2017

2017

Earlier work this paper cites.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805

Original

2018

Earlier work this paper cites.

J. Leike, D. Krueger, T. Everitt, M. Martic, V. Maini, and S. Legg, “Scalable agent alignment via reward modeling: A research direction. arxiv 2018,” arXiv preprint arXiv:1811.07871

Original

2018

Earlier work this paper cites.

G. Irving, P. Christiano, and D. Amodei, “Ai safety via debate,” arXiv preprint arXiv:1805.00899

Original

2018

Earlier work this paper cites.

Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and J. Li, “Boosting adversarial attacks with momentum,” 2018

2018

Earlier work this paper cites.

E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh, “Universal adversarial triggers for attacking and analyzing nlp,” arXiv preprint arXiv:1908.07125

Original

2019

Earlier work this paper cites.

D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irving, “Fine-tuning language models from human preferences,” arXiv preprint arXiv:1909.08593

Original

2019

Earlier work this paper cites.

B. Hancock, A. Bordes, P.-E. Mazare, and J. Weston, “Learning from dialogue after deployment: Feed yourself, chatbot!,” arXiv preprint arXiv:1901.05415

Original

2019

Earlier work this paper cites.

I. Provilkov, D. Emelianenko, and E. Voita, “Bpe-dropout: Simple and effective subword regularization,” arXiv preprint arXiv:1910.13267

Original

2019

Earlier work this paper cites.

T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg, “Badnets: Evaluating backdooring attacks on deep neural networks,” IEEE Access

2019

Earlier work this paper cites.

C. Xie, Z. Zhang, Y. Zhou, S. Bai, J. Wang, Z. Ren, and A. Yuille, “Improving transferability of adversarial examples with input diversity,” 2019

2019

Earlier work this paper cites.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al

2020

Earlier work this paper cites.

C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research

2020

Earlier work this paper cites.

T. Schick and H. Schütze, “Exploiting cloze questions for few shot text classification and natural language inference,” arXiv preprint arXiv:2001.07676

Original

2020

Earlier work this paper cites.

T. Gao, A. Fisch, and D. Chen, “Making pre-trained language models better few-shot learners,” arXiv preprint arXiv:2012.15723

Original

2020

Earlier work this paper cites.

T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh, “Autoprompt: Eliciting knowledge from language models with automatically generated prompts,” arXiv preprint arXiv:2010.15980

Original

2020

Earlier work this paper cites.

L. Shu, A. Papangelis, Y.-C. Wang, G. Tur, H. Xu, Z. Feizollahi, B. Liu, and P. Molino, “Controllable text generation with focused variation,” arXiv preprint arXiv:2009.12046

Original

2020

Earlier work this paper cites.

N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano, “Learning to summarize with human feedback,” Advances in Neural Information Processing Systems

2020

Earlier work this paper cites.

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al

2021

Earlier work this paper cites.

A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever, “Zero-shot text-to-image generation,” in International conference on machine learning

2021

Earlier work this paper cites.

N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, et al

2021

Earlier work this paper cites.

L. Reynolds and K. McDonell, “Prompt programming for large language models: Beyond the few-shot paradigm,” in Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

2021

Earlier work this paper cites.

X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv preprint arXiv:2101.00190

Original

2021

Earlier work this paper cites.

G. Qin and J. Eisner, “Learning how to ask: Querying lms with mixtures of soft prompts,” arXiv preprint arXiv:2104.06599

Original

2021

Earlier work this paper cites.

J. Wu, L. Ouyang, D. M. Ziegler, N. Stiennon, R. Lowe, J. Leike, and P. Christiano, “Recursively summarizing books with human feedback,” arXiv preprint arXiv:2109.10862

Original

2021

Earlier work this paper cites.

J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al

2022

Earlier work this paper cites.

Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al

Original

2022

Earlier work this paper cites.

Y. Li, Y. Jiang, Z. Li, and S.-T. Xia, “Backdoor learning: A survey,” IEEE Transactions on Neural Networks and Learning Systems

2022

Earlier work this paper cites.

J. Zhang, Q. Yi, and J. Sang, “Towards adversarial attack on vision-language pre-training models,” 2022

2022

Earlier work this paper cites.

X. Yang, Y. Dong, T. Pang, H. Su, and J. Zhu, “Boosting transferability of targeted adversarial examples via hierarchical generative networks,” 2022

2022

Earlier work this paper cites.

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al

Original

2023

Earlier work this paper cites.

A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al

2023

Earlier work this paper cites.

E. Shayegani, M. A. A. Mamun, Y. Fu, P. Zaree, Y. Dong, and N. Abu-Ghazaleh, “Survey of vulnerabilities in large language models revealed by adversarial attacks,” arXiv preprint arXiv:2310.10844

Original

2023

Earlier work this paper cites.

A. Esmradi, D. W. Yip, and C. F. Chan, “A comprehensive survey of attack techniques, implementation, and mitigation strategies in large language models,” arXiv preprint arXiv:2312.10982

Original

2023

Earlier work this paper cites.

A. Rao, S. Vashistha, A. Naik, S. Aditya, and M. Choudhury, “Tricking llms into disobedience: Understanding, analyzing, and preventing jailbreaks,” arXiv preprint arXiv:2305.14965

Original

2023

Earlier work this paper cites.

H. Liu, M. Chaudhary, and H. Wang, “Towards trustworthy and aligned machine learning: A data-centric survey with causality perspectives,” arXiv preprint arXiv:2307.16851

Original

2023

Earlier work this paper cites.

Ö. AYDIN, “Google bard generated literature review: metaverse,” Journal of AI

2023

Earlier work this paper cites.

W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, et al

2023

Earlier work this paper cites.

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al

Original

2023

Earlier work this paper cites.

P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. N. Pre-train, “prompt, and predict: A systematic survey of prompting methods in natural language processing., 2023, 55,” DOI: https://doi. org/10.1145/3560815

2023

Earlier work this paper cites.

A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” arXiv preprint arXiv:2307.15043

Original

2023

Earlier work this paper cites.

S. Zhu, R. Zhang, B. An, G. Wu, J. Barrow, Z. Wang, F. Huang, A. Nenkova, and T. Sun, “Autodan: Automatic and interpretable adversarial attacks on large language models,” arXiv preprint arXiv:2310.15140

Original

2023

Earlier work this paper cites.

D. Yao, J. Zhang, I. G. Harris, and M. Carlsson, “Fuzzllm: A novel and universal fuzzing framework for proactively discovering jailbreak vulnerabilities in large language models,” arXiv preprint arXiv:2309.05274

Original

2023

Earlier work this paper cites.

J. Yu, X. Lin, and X. Xing, “Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts,” arXiv preprint arXiv:2309.10253

Original

2023

Earlier work this paper cites.

X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “" do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” arXiv preprint arXiv:2308.03825

Original

2023

Earlier work this paper cites.

H. Li, D. Guo, W. Fan, M. Xu, and Y. Song, “Multi-step jailbreaking privacy attacks on chatgpt,” arXiv preprint arXiv:2304.05197

Original

2023

Earlier work this paper cites.

P. Ding, J. Kuang, D. Ma, X. Cao, Y. Xian, J. Chen, and S. Huang, “A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily,” arXiv preprint arXiv:2311.08268

Original

2023

Earlier work this paper cites.

P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong, “Jailbreaking black box large language models in twenty queries,” arXiv preprint arXiv:2310.08419

Original

2023

Earlier work this paper cites.

E. Jones, A. Dragan, A. Raghunathan, and J. Steinhardt, “Automatically auditing large language models via discrete optimization,” in International Conference on Machine Learning

2023

Earlier work this paper cites.

R. Lapid, R. Langberg, and M. Sipper, “Open sesame! universal black box jailbreaking of large language models,” arXiv preprint arXiv:2309.01446

Original

2023

Earlier work this paper cites.

X. Liu, N. Xu, M. Chen, and C. Xiao, “Autodan: Generating stealthy jailbreak prompts on aligned large language models,” arXiv preprint arXiv:2310.04451

Original

2023

Cited alongside, same era.

Y. Huang, S. Gupta, M. Xia, K. Li, and D. Chen, “Catastrophic jailbreak of open-source llms via exploiting generation,” arXiv preprint arXiv:2310.06987

Original

2023

Cited alongside, same era.

Z. Wei, Y. Wang, and Y. Wang, “Jailbreak and guard aligned language models with only few in-context demonstrations,” arXiv preprint arXiv:2310.06387

Original

2023

Cited alongside, same era.

S. Schulhoff, J. Pinto, A. Khan, L.-F. Bouchard, C. Si, S. Anati, V. Tagliabue, A. L. Kost, C. Carnahan, and J. Boyd-Graber, “Ignore this title and hackaprompt: Exposing systemic vulnerabilities of llms through a global scale prompt hacking competition,” arXiv preprint arXiv:2311.16119

Original

2023

Cited alongside, same era.

X. Qi, Y. Zeng, T. Xie, P.-Y. Chen, R. Jia, P. Mittal, and P. Henderson, “Fine-tuning aligned language models compromises safety, even when users do not intend to!,” arXiv preprint arXiv:2310.03693

Original

2023

Later among the works it cites.

X. Chen, S. Tang, R. Zhu, S. Yan, L. Jin, Z. Wang, L. Su, X. Wang, and H. Tang, “The janus interface: How fine-tuning in large language models amplifies the privacy risks,” arXiv preprint arXiv:2310.15469

Original

2023

Later among the works it cites.

L. Lin, H. Mu, Z. Zhai, M. Wang, Y. Wang, R. Wang, J. Gao, Y. Zhang, W. Che, T. Baldwin, et al

Original

2024

Closest in time.

Q. Ren, C. Gao, J. Shao, J. Yan, X. Tan, W. Lam, and L. Ma, “Exploring safety generalization challenges of large language models via code,” arXiv preprint arXiv:2403.07865

Original

2024

Closest in time.