Fetching the paper…

Reading the bibliography…

2024

Goal-guided Generative Prompt Injection Attack on Large Language Models

Zhang, Chong, Jin, Mingyu, Yu, Qinkai et al.

Original
PDF
Post on X

Understand

Current large language models (LLMs) provide a strong foundation for large-scale user-oriented natural language tasks.

A large number of users can easily inject adversarial text or instructions through the user interface, thus causing LLMs model security challenges.
Although there is currently a large amount of research on prompt injection attacks, most of these black-box attacks use heuristic strategies.
It is unclear how these heuristic strategies relate to the success rate of attacks and thus effectively improve model robustness.

Goal-guided Generative Prompt Injection Attack on Large Language Models · Around

Built on

arXiv:1901.11196
Original
J. Wei, K. Zou, Eda: Easy data augmentation techniques for boosting performance on text classification tasks (2019) · 1901
Earlier work this paper cites.
arXiv:1908.07125
Original
E. Wallace, S. Feng, N. Kandpal, M. Gardner, S. Singh, Universal adversarial triggers for attacking and analyzing nlp (2021) · 1908
Earlier work this paper cites.
arXiv:2005.05909
Original
J. X. Morris, E. Lifland, J. Y. Yoo, J. Grigsby, D. Jin, Y. Qi, Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp (2020) · 2005
Earlier work this paper cites.
arXiv:2010.15980
Original
T. Shin, Y. Razeghi, R. L. L. I. au2, E. Wallace, S. Singh, Autoprompt: Eliciting knowledge from language models with automatically generated prompts (2020) · 2010
Earlier work this paper cites.
I. J. Goodfellow, J. Shlens, C. Szegedy, Explaining and harnessing adversarial examples, arXiv preprint arXiv:1412.6572 (2014)
Original
2014
Earlier work this paper cites.
S.-M. Moosavi-Dezfooli, A. Fawzi, P. Frossard, Deepfool: a simple and accurate method to fool deep neural networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, p.
2016
Earlier work this paper cites.
T. Gu, B. Dolan-Gavitt, S. Garg, Badnets: Identifying vulnerabilities in the machine learning model supply chain, arXiv preprint arXiv:1708.06733 (2017)
Original
2017
Earlier work this paper cites.
N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, A. Swami, Practical black-box attacks against machine learning, in: ACCC, 2017, p.
2017
Earlier work this paper cites.
J. Li, S. Ji, T. Du, B. Li, T. Wang, Textbugger: Generating adversarial text against real-world applications, arXiv preprint arXiv:1812.05271 (2018)
Original
2018
Earlier work this paper cites.
arXiv:1712.06751
Original
J. Ebrahimi, A. Rao, D. Lowd, D. Dou, Hotflip: White-box adversarial examples for text classification (2018) · 2018
Earlier work this paper cites.
M. T. Ribeiro, S. Singh, C. Guestrin, Semantically equivalent adversarial rules for debugging NLP models, in: ACL, 2018, p.
2018
Earlier work this paper cites.
P. Rajpurkar, R. Jia, P. Liang, Know what you don’t know: Unanswerable questions for squad, arXiv preprint arXiv:1806.03822 (2018)
Original
2018
Earlier work this paper cites.

Similar

J. Gao, J. Lanchantin, M. L. Soffa, Y. Qi, Black-box generation of adversarial text sequences to evade deep learning classifiers, in: SPW, 2018, p.
2018
Cited alongside, same era.
D. Jin, Z. Jin, J. T. Zhou, P. Szolovits, Is bert really robust? a strong baseline for natural language attack on text classification and entailment, in: AAAI, 2020, p.
2020
Cited alongside, same era.
L. Li, R. Ma, Q. Guo, X. Xue, X. Qiu, BERT-ATTACK: adversarial attack against BERT using BERT, in: EMNLP, 2020, p.
2020
Cited alongside, same era.
A. Radford, J. Wu, R. Child, D. Luan, A. B. Santoro, S. Chaplot, A. Patra, I. Sutskever, Chatgpt: A language model for conversational agents, OpenAI (2020)
2020
Cited alongside, same era.
M. T. Ribeiro, T. Wu, C. Guestrin, S. Singh, Beyond accuracy: Behavioral testing of nlp models with checklist, arXiv preprint arXiv:2005.04118 (2020)
Original

Then

OpenAI, Gpt-4 technical report (2023)
2023
Later among the works it cites.
T. Kaufmann, P. Weng, V. Bengs, E. Hüllermeier, A survey of reinforcement learning from human feedback, arXiv preprint arXiv:2312.14925 (2023)
Original
2023
Later among the works it cites.
arXiv:2310.10844
Original
E. Shayegani, M. A. A. Mamun, Y. Fu, P. Zaree, Y. Dong, N. Abu-Ghazaleh, Survey of vulnerabilities in large language models revealed by adversarial attacks (2023) · 2023
Later among the works it cites.
S. Goyal, S. Doddapaneni, M. M. Khapra, B. Ravindran, A survey of adversarial defenses and robustness in nlp, ACM Comput. Surv. (2023)
2023
Later among the works it cites.
arXiv:2303.04381
Original
E. Jones, A. Dragan, A. Raghunathan, J. Steinhardt, Automatically auditing large language models via discrete optimization (2023) · 2023

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2020

Cited alongside, same era.

arXiv:2104.13733

C. Guo, A. Sablayrolles, H. Jégou, D. Kiela, Gradient-based adversarial attacks against text transformers (2021) · 2021

Cited alongside, same era.

C. L. Canonne, G. Kamath, T. Steinke, The discrete gaussian for differential privacy (2021)

2021

Cited alongside, same era.

D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, J. Steinhardt, Measuring mathematical problem solving with the math dataset, NeurIPS (2021)

2021

Cited alongside, same era.

arXiv:2211.09527

F. Perez, I. Ribeiro, Ignore previous prompt: Attack techniques for language models (2022) · 2022

Cited alongside, same era.

E. Wallace, S. Feng, N. Kandpal, M. Gardner, S. Singh, Benchmarking language models’ robustness to semantic perturbations, in: NAACL, 2022, p.

2022

Cited alongside, same era.

Y. Chang, M. Narang, H. Suzuki, G. Cao, J. Gao, Y. Bisk, Webqa: Multihop and multimodal qa, in: CVPR, 2022, pp. 16495–16504

2022

Cited alongside, same era.

Later among the works it cites.

I. R. McKenzie, A. Lyzhov, M. Pieler, A. Parrish, A. Mueller, A. Prabhu, E. McLean, A. Kirtland, A. Ross, A. Liu, et al., Inverse scaling: When bigger isn’t better, arXiv preprint arXiv:2306.09479 (2023)

2023

Later among the works it cites.

arXiv:2306.05499

Y. Liu, G. Deng, Y. Li, K. Wang, T. Zhang, Y. Liu, H. Wang, Y. Zheng, Y. Liu, Prompt injection attack against llm-integrated applications (2023) · 2023

Later among the works it cites.

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al., Llama: Open and efficient foundation language models. corr, abs/2302.13971, 2023. doi: 10.48550, arXiv preprint arXiv.2302.13971 (2023)

2023

Later among the works it cites.

arXiv:2302.06066

T. Brown, J. Chinchilla, Q. V. Le, B. Mann, A. Roy, D. Saxton, E. Wei, M. Ziegler, Gsm8k: A large-scale dataset for language model training (2023) · 2023

Later among the works it cites.

Z. Zhou, Q. Wang, M. Jin, J. Yao, J. Ye, W. Liu, W. Wang, X. Huang, K. Huang, Mathattack: Attacking large language models towards math solving ability, arXiv preprint arXiv:2309.01686 (2023)

2023

Later among the works it cites.

K. Zhu, J. Wang, J. Zhou, Z. Wang, H. Chen, Y. Wang, L. Yang, W. Ye, N. Z. Gong, Y. Zhang, et al., Promptbench: Towards evaluating the robustness of large language models on adversarial prompts, arXiv preprint arXiv:2306.04528 (2023)

2023

Later among the works it cites.