Fetching the paper…

Reading the bibliography…

2023

Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts

Wu, Yuanwei, Li, Xiang, Liu, Yixin et al.

Original
PDF
Post on X

Understand

Existing work on jailbreak Multimodal Large Language Models (MLLMs) has focused primarily on adversarial examples in model inputs, with less attention to vulnerabilities, especially in model API.

To fill the research gap, we carry out the following work: 1) We discover a system prompt leakage vulnerability in GPT-4V.
Through carefully designed dialogue, we successfully extract the internal system prompts of GPT-4V.
This finding indicates potential exploitable security risks in MLLMs; 2) Based on the acquired system prompts, we propose a novel MLLM jailbreaking attack method termed SASP (Self-Adversarial Attack via System Prompt).

Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts · Around

Built on

Deep learning face attributes in the wild
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015 · 2015
Earlier work this paper cites.
Autoprompt: Eliciting knowledge from language models with automatically generated prompts
Original
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV au2, Eric Wallace, and Sameer Singh. 2020 · 2020
Earlier work this paper cites.
Flamingo: a visual language model for few-shot learning
Original
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan. 2022 · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Original
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022 · 2022
Earlier work this paper cites.
Celebrity face image dataset
VISHESH THAKUR. 2022 · 2022
Earlier work this paper cites.
Image hijacks: Adversarial images can control generative models at runtime
Original
Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons. 2023 · 2023
Earlier work this paper cites.
Minigpt-v2: large language model as a unified interface for vision-language multi-task learning
Original
Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechu Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023 · 2023
Earlier work this paper cites.
How robust is google’s bard to adversarial image attacks?
Original
Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, and Jun Zhu. 2023 · 2023
Earlier work this paper cites.

Similar

Behind the scenes
Surav Shrestha Dustin Miller, Michael Skyba. 2023 · 2023
Cited alongside, same era.
Pretraining language models with human preferences
Original
Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Bhalerao, Christopher L. Buckley, Jason Phang, Samuel R. Bowman, and Ethan Perez. 2023 · 2023
Cited alongside, same era.
Self-refine: Iterative refinement with self-feedback
Original
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023 · 2023
Cited alongside, same era.
Is a prompt and a few samples all you need? using gpt-4 for data augmentation in low-resource classification tasks
Original
Anders Giovanni Møller, Jacob Aarup Dalsgaard, Arianna Pera, and Luca Maria Aiello. 2023 · 2023
Cited alongside, same era.
Dera: Enhancing large language model completions with dialog-enabled resolving agents

Then

Jailbroken: How does llm safety training fail?
Original
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023 · 2023
Closest in time.
Large language models as optimizers
Original
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2023 · 2023
Closest in time.
mplug-owl: Modularization empowers large language models with multimodality
Original
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qian Qi, Ji Zhang, and Fei Huang. 2023 · 2023
Closest in time.
Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts
Original
Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2023 · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Varun Nair, Elliot Schumacher, Geoffrey Tso, and Anitha Kannan. 2023 · 2023

Cited alongside, same era.

Gpt models - openai api

OpenAI. 2022 · 2023

Cited alongside, same era.

Gpt-4v(ision) system card

OpenAI. 2023 · 2023

Cited alongside, same era.

Promptor: A conversational and autonomous prompt generation agent for intelligent text entry techniques

Junxiao Shen, John J Dudley, Jingyao Zheng, Bill Byrne, and Per Ola Kristensson. 2023 · 2023

Cited alongside, same era.

Training a helpful and harmless assistant with reinforcement learning from human feedback

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. 2022a

Cited in the paper.

Constitutional ai: Harmlessness from ai feedback

Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. 2022b

Cited in the paper.

Jailbreaker: Automated jailbreak across multiple large language model chatbots

Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2023a

Cited in the paper.

Masterkey: Automated jailbreak across multiple large language model chatbots

Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2023b

Cited in the paper.

System-level natural language feedback

Weizhe Yuan, Kyunghyun Cho, and Jason Weston. 2023 · 2023

Closest in time.

Prompts should not be seen as secrets: Systematically measuring prompt extraction attack success

Yiming Zhang and Daphne Ippolito. 2023 · 2023

Closest in time.

Universal and Transferable Adversarial Attacks on Aligned Language Models

Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023 · 2023

Closest in time.