Transferability of Adversarial Attacks in Video-based MLLMs: A Cross-modal Image-to-Video Approach

Understand

Video-based multimodal large language models (V-MLLMs) have shown vulnerability to adversarial examples in video-text multimodal tasks.

• However, the transferability of adversarial videos to unseen models - a common and practical real-world scenario - remains unexplored.
• In this paper, we pioneer an investigation into the transferability of adversarial video samples across V-MLLMs.
• We find that existing adversarial attack methods face significant limitations when applied in black-box settings for V-MLLMs, which we attribute to the following shortcomings: (1) lacking generalization in perturbing video features, (2) focusing only on sparse key-frames, and (3) failing to integrate multimodal information.