Jailbreak Large Vision-Language Models Through Multi-Modal Linkage

Natural language processing with Python: analyzing text with the natural language toolkit

Steven Bird, Ewan Klein, and Edward Loper. 2009 · 2009

Earlier work this paper cites.

Intriguing properties of neural networks

Original

C Szegedy. 2013 · 2013

Earlier work this paper cites.

Explaining and harnessing adversarial examples

Original

Ian J Goodfellow. 2014 · 2014

Earlier work this paper cites.

Training a helpful and harmless assistant with reinforcement learning from human feedback

Original

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022 · 2022

Earlier work this paper cites.

A survey of deep learning for mathematical reasoning

Original

Pan Lu, Liang Qiu, Wenhao Yu, Sean Welleck, and Kai-Wei Chang. 2022 · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 · 2022

Earlier work this paper cites.

Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

Original

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023 · 2023

Earlier work this paper cites.

Jailbreaking black box large language models in twenty queries

Original

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023 · 2023

Earlier work this paper cites.

Minigpt-v2: large language model as a unified interface for vision-language multi-task learning

Original

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023 · 2023

Earlier work this paper cites.

Opencompass: A universal evaluation platform for foundation models

OpenCompass Contributors. 2023 · 2023

Earlier work this paper cites.

How robust is google’s bard to adversarial image attacks?

Original

Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, and Jun Zhu. 2023 · 2023

Earlier work this paper cites.

Figstep: Jailbreaking large vision-language models via typographic visual prompts

Original

Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023 · 2023

Earlier work this paper cites.

Llama guard: Llm-based input-output safeguard for human-ai conversations

Original

Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023 · 2023

Earlier work this paper cites.

Jailbreaking chatgpt via prompt engineering: An empirical study

Original

Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, Kailong Wang, and Yang Liu. 2023 · 2023

Cited alongside, same era.

Llama 2 acceptable use policy

Meta. 2023 · 2023

Cited alongside, same era.

Cogvlm: Visual expert for pretrained language models

Original

Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. 2023 · 2023

Cited alongside, same era.

A mutation-based method for multi-modal jailbreaking attack detection

Original

Xiaoyu Zhang, Cen Zhang, Tianlin Li, Yihao Huang, Xiaojun Jia, Xiaofei Xie, Yang Liu, and Chao Shen. 2023 · 2023

Cited alongside, same era.

Universal and transferable adversarial attacks on aligned language models

Fight back against jailbreaking via prompt adversarial tuning

Yichuan Mo, Yuji Wang, Zeming Wei, and Yisen Wang. 2024 · 2024

Closest in time.

Jailbreaking attack against multimodal large language model

Original

Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. 2024 · 2024

Closest in time.

Gpt-4 technical report

Original

OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and Shyamal Anadkat et al. 2024 · 2024

Closest in time.

MLLM-protector: Ensuring MLLM’s safety without hurting performance

Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang. 2024 · 2024

Closest in time.

Visual adversarial examples jailbreak aligned large language models

Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024 · 2024

Closest in time.