$\textit{MMJ-Bench}$: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models

Random gradient-free minimization of convex functions

Yurii Nesterov and Vladimir Spokoiny. 2017 · 2017

Earlier work this paper cites.

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Earlier work this paper cites.

Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023 · 2023

Earlier work this paper cites.

Image hijacks: Adversarial images can control generative models at runtime

Original

Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons. 2023 · 2023

Earlier work this paper cites.

Instructblip: Towards general-purpose vision-language models with instruction tuning

Wenliang Dai, Junnan Li, DONGXU LI, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2023 · 2023

Earlier work this paper cites.

Figstep: Jailbreaking large vision-language models via typographic visual prompts

Original

Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023 · 2023

Earlier work this paper cites.

Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models

Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2023 · 2023

Earlier work this paper cites.

A survey on multimodal large language models

Original

Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023 · 2023

Earlier work this paper cites.

Mm-vet: Evaluating large multimodal models for integrated capabilities

Original

Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2023 · 2023

Cited alongside, same era.

A mutation-based method for multi-modal jailbreaking attack detection

Original

Xiaoyu Zhang, Cen Zhang, Tianlin Li, Yihao Huang, Xiaojun Jia, Xiaofei Xie, Yang Liu, and Chao Shen. 2023 · 2023

Cited alongside, same era.

Minigpt-4: Enhancing vision-language understanding with advanced large language models

Original

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023 · 2023

Cited alongside, same era.

Universal and transferable adversarial attacks on aligned language models

Original

Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023 · 2023

Cited alongside, same era.

Jailbreaking attack against multimodal large language model

Original

Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. 2024 · 2024

Closest in time.

Visual adversarial examples jailbreak aligned large language models

Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024 · 2024

Closest in time.

Jailbroken: How does llm safety training fail?

Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2024 · 2024

Closest in time.

Defending jailbreak attack in vlms via cross-modality information detector

Original

Yue Xu, Xiuyuan Qi, Zhan Qin, and Wenjie Wang. 2024 · 2024

Closest in time.

Mmt-bench: A comprehensive multimodal benchmark for evaluating large vision-language models towards multitask agi

Original

Kaining Ying, Fanqing Meng, Jin Wang, Zhiqian Li, Han Lin, Yue Yang, Hao Zhang, Wenbo Zhang, Yuqi Lin, Shuo Liu, et al. 2024 · 2024