Fetching the paper…
Reading the bibliography…
In this paper we argue that a jailbroken GenAI model can cause substantial harm to GenAI-powered applications and facilitate PromptWare, a new type of attack that flips the GenAI model's behavior from serving an application to attacking it.
Ignore previous prompt: Attack techniques for language models
Fábio Perez and Ian Ribeiro · 2022
Earlier work this paper cites.
Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection
Sahar Abdelnabi, Kai Greshake, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz · 2023
Earlier work this paper cites.
(ab) using images and sounds for indirect instruction injection in multi-modal llms
Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi, and Vitaly Shmatikov · 2023
Earlier work this paper cites.
Are aligned neural networks adversarially aligned?
Nicholas Carlini, Milad Nasr, Christopher A Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, et al · 2023
Earlier work this paper cites.
Jailbreaking black box large language models in twenty queries
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong · 2023
Earlier work this paper cites.
Jailbreaker: Automated jailbreak across multiple large language model chatbots
Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu · 2023
Earlier work this paper cites.
Getting pwn’d by ai: Penetration testing with large language models
Andreas Happe and Jürgen Cito · 2023
Earlier work this paper cites.
An llm compiler for parallel function calling
Sehoon Kim, Suhong Moon, Ryan Tabrizi, Nicholas Lee, Michael W Mahoney, Kurt Keutzer, and Amir Gholami · 2023
Earlier work this paper cites.
Demystifying rce vulnerabilities in llm-integrated apps
Tong Liu, Zizhuang Deng, Guozhu Meng, Yuekang Li, and Kai Chen · 2023
Earlier work this paper cites.
Prompt injection attacks and defenses in llm-integrated applications
Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong · 2023
Earlier work this paper cites.
Scalable extraction of training data from (production) language models
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee · 2023
Cited alongside, same era.
Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models
Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim · 2023
Cited alongside, same era.
Rewoo: Decoupling reasoning from observations for efficient augmented language models
Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, and Dongkuan Xu · 2023
Cited alongside, same era.
Universal and transferable adversarial attacks on aligned language models
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson · 2023
Cited alongside, same era.
Agent smith: A single image can jailbreak one million multimodal llm agents exponentially fast
Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin · 2024
Closest in time.
Generative ai for pentesting: the good, the bad, the ugly
Eric Hilario, Sami Azam, Jawahar Sundaram, Khwaja Imran Mohammed, and Bharanidharan Shanmugam · 2024
Closest in time.
Prompt stealing attacks against large language models
Zeyang Sha and Yang Zhang · 2024
Closest in time.
Machine against the rag: Jamming retrieval-augmented generation with blocker documents
Avital Shafran, Roei Schuster, and Vitaly Shmatikov · 2024
Closest in time.
What was your prompt? a remote keylogging attack on ai assistants
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Divyansh Agarwal, Alexander R Fabbri, Philippe Laban, Shafiq Joty, Caiming Xiong, and Chien-Sheng Wu · 2024
Cited alongside, same era.
Stealing part of a production language model
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, et al · 2024
Cited alongside, same era.
Here comes the ai worm: Unleashing zero-click worms that target genai-powered applications, 2024
Stav Cohen, Ron Bitton, and Ben Nassi · 2024
Cited alongside, same era.
Llm agents can autonomously exploit one-day vulnerabilities
Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang · 2024
Cited alongside, same era.
Llm agents can autonomously hack websites
Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang · 2024
Cited alongside, same era.
Teams of llm agents can exploit zero-day vulnerabilities
Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang · 2024
Cited alongside, same era.
Roy Weiss, Daniel Ayzenshteyn, Guy Amit, and Yisroel Mirsky · 2024
Closest in time.
A new era in llm security: Exploring security concerns in real-world llm-based systems
Fangzhou Wu, Ning Zhang, Somesh Jha, Patrick McDaniel, and Chaowei Xiao · 2024
Closest in time.
Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models
Jiaqi Xue, Mengxin Zheng, Yebowen Hu, Fei Liu, Xun Chen, and Qian Lou · 2024
Closest in time.
Prsa: Prompt reverse stealing attacks against large language models
Yong Yang, Xuhong Zhang, Yi Jiang, Xi Chen, Haoyu Wang, Shouling Ji, and Zonghui Wang · 2024
Closest in time.
Effective prompt extraction from language models, 2024
Yiming Zhang, Nicholas Carlini, and Daphne Ippolito · 2024
Closest in time.
Poisonedrag: Knowledge poisoning attacks to retrieval-augmented generation of large language models
Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia · 2024
Closest in time.