Built on

C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” 2023. [Online]. Available: https://arxiv.org/abs/1910.10683
Original
1910
Earlier work this paper cites.
D. Lee and M. Yannakakis, “Principles and methods of testing finite state machines-a survey,” Proceedings of the IEEE , vol. 84, no. 8, pp. 1090–1123, 1996
1996
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is All you Need,” in Advances in Neural Information Processing Systems , 2017, pp. 5998–6008. [Online]. Available: http://arxiv.org/abs/1706.03762
Original
2017
Earlier work this paper cites.
M. Iyyer, J. Wieting, K. Gimpel, and L. Zettlemoyer, “Adversarial Example Generation with Syntactically Controlled Paraphrase Networks,” arXiv preprint arXiv:1804.06059 , 2018
Original
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) , 2019, pp. 4171–4186
2019
Earlier work this paper cites.
D. J. Fremont, T. Dreossi, S. Ghosh, X. Yue, A. L. Sangiovanni-Vincentelli, and S. A. Seshia, “Scenic: A language for scenario specification and scene generation,” in Proceedings of the 40th ACM SIGPLAN conference on programming language design and implementation , 2019, pp. 63–78
2019
Earlier work this paper cites.
F. Qi, Y. Chen, M. Li, Y. Yao, Z. Liu, and M. Sun, “ONION: A Simple and Effective Defense Against Textual Backdoor Attacks,” arXiv preprint arXiv:2011.10369 , 2020
Original
2020
Earlier work this paper cites.
F. Qi, M. Li, Y. Chen, Z. Zhang, Z. Liu, Y. Wang, and M. Sun, “Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger,” arXiv preprint arXiv:2105.12400 , 2021
Original
2021
Earlier work this paper cites.
S. Li, H. Liu, T. Dong, B. Z. H. Zhao, M. Xue, H. Zhu, and J. Lu, “Hidden Backdoors in Human-Centric Language Models,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , 2021, pp. 3123–3140
2021
Earlier work this paper cites.
F. Perez and I. Ribeiro, “Ignore Previous Prompt: Attack Techniques for Language Models,” arXiv preprint arXiv:2211.09527 , 2022
Original
2022
Earlier work this paper cites.
W. Zhang, Y. Deng, B. Liu, S. J. Pan, and L. Bing, “Sentiment Analysis in the Era of Large Language Models: A Reality Check,” arXiv preprint arXiv:2305.15005 , 2023
Original
2023
Earlier work this paper cites.
OpenAI, “ChatGPT (Feb 20 Version),” 2023. [Online]. Available: https://openai.com/chatgpt
2023
Earlier work this paper cites.
H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, and A. Mian, “A Comprehensive Overview of Large Language Models,” arXiv preprint arXiv:2307.06435 , 2023
Original
2023
Earlier work this paper cites.
H. Huang, Z. Zhao, M. Backes, Y. Shen, and Y. Zhang, “Composite Backdoor Attacks Against Large Language Models,” arXiv preprint arXiv:2310.07676 , 2023
Original
2023
Earlier work this paper cites.
J. Shi, Y. Liu, P. Zhou, and L. Sun, “BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT,” arXiv preprint arXiv:2304.12298 , 2023
Original
2023
Earlier work this paper cites.
J. Wang, J. Wu, M. Chen, Y. Vorobeychik, and C. Xiao, “RLHFPoison: Reward Poisoning Attack for Reinforcement Learning with Human Feedback in Large Language Models,” arXiv preprint arXiv:2311.09641 , 2023
Original
2023
Earlier work this paper cites.
J. Xue, M. Zheng, T. Hua, Y. Shen, Y. Liu, L. Bölöni, and Q. Lou, “TrojLLM: A Black-box Trojan Prompt Attack on Large Language Models,” Advances in Neural Information Processing Systems , vol. 36, pp. 65 665–65 677, 2023
2023
Earlier work this paper cites.
T. Dong, M. Xue, G. Chen, R. Holland, Y. Meng, S. Li, Z. Liu, and H. Zhu, “The Philosopher’s Stone: Trojaning Plugins of Large Language Models,” arXiv preprint arXiv:2312.00374 , 2023
Original
2023
Earlier work this paper cites.
N. Gu, P. Fu, X. Liu, Z. Liu, Z. Lin, and W. Wang, “A Gradient Control Method for Backdoor Attacks on Parameter-Efficient Tuning,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 3508–3520
2023
Earlier work this paper cites.
H. Wang and K. Shu, “Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignment,” arXiv preprint arXiv:2311.09433 , 2023
Original
2023
Earlier work this paper cites.
A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How Does LLM Safety Training Fail?” Advances in Neural Information Processing Systems , vol. 36, pp. 80 079–80 110, 2023
2023
Earlier work this paper cites.
J. Yu, X. Lin, Z. Yu, and X. Xing, “GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts,” arXiv preprint arXiv:2309.10253 , 2023
Original
2023
Earlier work this paper cites.
P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong, “Jailbreaking Black Box Large Language Models in Twenty Queries,” arXiv preprint arXiv:2310.08419 , 2023
Original
2023
Earlier work this paper cites.
A. Mehrotra, M. Zampetakis, P. Kassianik, B. Nelson, H. Anderson, Y. Singer, and A. Karbasi, “Tree of Attacks: Jailbreaking Black-Box LLMs Automatically,” 2023
2023
Earlier work this paper cites.
Z.-X. Yong, C. Menghini, and S. H. Bach, “Low-resource Languages Jailbreak GPT-4,” arXiv preprint arXiv:2310.02446 , 2023
Original
2023
Earlier work this paper cites.
Y. Deng, W. Zhang, S. J. Pan, and L. Bing, “Multilingual Jailbreak Challenges in Large Language Models,” arXiv preprint arXiv:2310.06474 , 2023
Original
2023
Earlier work this paper cites.
A. Robey, E. Wong, H. Hassani, and G. J. Pappas, “Smoothllm: Defending Large Language Models Against Jailbreaking Attacks,” arXiv preprint arXiv:2310.03684 , 2023
Original
2023
Earlier work this paper cites.
R. Shah, S. Pour, A. Tagade, S. Casper, J. Rando et al. , “Scalable and Transferable Black-box Jailbreaks for Language Models via Persona Modulation,” arXiv preprint arXiv:2311.03348 , 2023
Original
2023
Earlier work this paper cites.
N. Jain, A. Schwarzschild, Y. Wen, G. Somepalli, J. Kirchenbauer, P.-y. Chiang, M. Goldblum, A. Saha, J. Geiping, and T. Goldstein, “Baseline Defenses for Adversarial Attacks Against Aligned Language Models,” arXiv preprint arXiv:2309.00614 , 2023
Original
2023
Earlier work this paper cites.

Similar

A. Mendes, “Ultimate ChatGPT Prompt Engineering Guide for General Users and Developers,” 2023. [Online]. Available: https://www.imaginarycloud.com/blog/chatgpt-prompt-engineering
2023
Cited alongside, same era.
“Sandwich defense,” 2023. [Online]. Available: {https://learnprompting.org/docs/prompt\_hacking/defensive\_measures/\\sandwich\_defense}
2023
Cited alongside, same era.
“Instruction defense,” 2023. [Online]. Available: \url{https://learnprompting.org/docs/prompt\_hacking/defensive\_measures/\\instruction}
2023
Cited alongside, same era.
G. Alon and M. Kamfonas, “Detecting Language Model Attacks with Perplexity,” arXiv preprint arXiv:2308.14132 , 2023
Original
2023
Cited alongside, same era.
E. Yudkowsky, “Using gpt: Eliezer against chatgpt jailbreaking,” 2023. [Online]. Available: \url{https://www.alignmentforum.org/posts/pNcFYZnPdXyL2RfgA/using-gpt-eliezer-against-chatgpt-jailbreaking}

Then

P. Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V. Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Tramer et al. , “JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models,” arXiv preprint arXiv:2404.01318 , 2024
Original
2024
Later among the works it cites.
Z. Chang, M. Li, Y. Liu, J. Wang, Q. Wang, and Y. Liu, “Play guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues,” arXiv preprint arXiv:2402.09091 , 2024
Original
2024
Later among the works it cites.
Y. Zeng, H. Lin, J. Zhang, D. Yang, R. Jia, and W. Shi, “How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2024, pp. 14 322–14 350
2024
Later among the works it cites.
Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and Benchmarking Prompt Injection Attacks and Defenses,” in 33rd USENIX Security Symposium (USENIX Security 24) , 2024, pp. 1831–1847

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2025

A Survey of Attacks on Large Language Models

Xu, Wenrui, Parhi, Keshab K.

Understand

Large language models (LLMs) and LLM-based agents have been widely deployed in a wide range of applications in the real world, including healthcare diagnostics, financial analysis, customer support, robotics, and autonomous driving, expanding their powerful capability of understanding, reasoning, and generating natural languages.

However, the wide deployment of LLM-based applications exposes critical security and reliability risks, such as the potential for malicious misuse, privacy leakage, and service disruption that weaken user trust and undermine societal safety.
This paper provides a systematic overview of the details of adversarial attacks targeting both LLMs and LLM-based agents.
These attacks are organized into three phases in LLMs: Training-Phase Attacks, Inference-Phase Attacks, and Availability & Integrity Attacks.