AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting

Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial Machine Learning at Scale. In: ICLR (2017)

2017

Earlier work this paper cites.

Mao, C., Chiquier, M., Wang, H., Yang, J., Vondrick, C.: Adversarial Attacks Are Reversible With Natural Supervision. In: ICCV (2021)

2021

Earlier work this paper cites.

Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. NeurIPS (2022)

2022

Earlier work this paper cites.

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

Original

2023

Earlier work this paper cites.

Awadalla, A., Gao, I., Gardner, J., Hessel, J., Hanafy, Y., Zhu, W., Marathe, K., Bitton, Y., Gadre, S., Sagawa, S., Jitsev, J., Kornblith, S., Koh, P.W., Ilharco, G., Wortsman, M., Schmidt, L.: OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models. arXiv preprint arXiv:2308.01390 (2023)

Original

2023

Earlier work this paper cites.

Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., Zhou, J.: Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv preprint arXiv:2308.12966 (2023)

Original

2023

Earlier work this paper cites.

Carlini, N., Nasr, M., Choquette-Choo, C.A., Jagielski, M., Gao, I., Awadalla, A., Koh, P.W., Ippolito, D., Lee, K., Tramer, F., Schmidt, L.: Are aligned neural networks adversarially aligned? In: NeurIPS (2023)

2023

Earlier work this paper cites.

Chen, J., Zhu, D., Shen, X., Li, X., Liu, Z., Zhang, P., Krishnamoorthi, R., Chandra, V., Xiong, Y., Elhoseiny, M.: MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310.09478 (2023)

Original

2023

Earlier work this paper cites.

Chen, K., Zhang, Z., Zeng, W., Zhang, R., Zhu, F., Zhao, R.: Shikra: Unleashing Multimodal LLM’s Referential Dialogue Magic. arXiv preprint arXiv:2306.15195 (2023)

Original

2023

Earlier work this paper cites.

Chen, Y., Sikka, K., Cogswell, M., Ji, H., Divakaran, A.: DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback. arXiv preprint arXiv:2311.10081 (2023)

Original

2023

Earlier work this paper cites.

Cheng, X., Cao, B., Ye, Q., Zhu, Z., Li, H., Zou, Y.: Ml-lmcl: Mutual learning and large-margin contrastive learning for improving asr robustness in spoken language understanding. In: Proc. of ACL Findings (2023)

2023

Earlier work this paper cites.

Cheng, X., Zhu, Z., Cao, B., Ye, Q., Zou, Y.: Mrrl: Modifying the reference via reinforcement learning for non-autoregressive joint multiple intent detection and slot filling. In: Proc. of EMNLP Findings (2023)

2023

Earlier work this paper cites.

Costa, J.C., Roxo, T., Proença, H., Inácio, P.R.M.: How Deep Learning Sees the World: A Survey on Adversarial Attacksn and Defenses. arXiv preprint arXiv:2305.10862 (2023)

Original

2023

Earlier work this paper cites.

Dong, Y., Chen, H., Chen, J., Fang, Z., Yang, X., Zhang, Y., Tian, Y., Su, H., Zhu, J.: How Robust is Google’s Bard to Adversarial Image Attacks? arXiv preprint arXiv:2309.11751 (2023)

Original

2023

Earlier work this paper cites.

Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Lin, X., Yang, J., Zheng, X., Li, K., Sun, X., Wu, Y., Ji, R.: MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. arXiv preprint arXiv:2306.13394 (2023)

Original

2023

Earlier work this paper cites.

Fu, C., Zhang, R., Wang, Z., Huang, Y., Zhang, Z., Qiu, L., Ye, G., Shen, Y., Zhang, M., Chen, P., Zhao, S., Lin, S., Jiang, D., Yin, D., Gao, P., Li, K., Li, H., Sun, X.: A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise. arXiv preprint arXiv:2312.12436 (2023)

Original

2023

Earlier work this paper cites.

Ge, J., Luo, H., Qian, S., Gan, Y., Fu, J., Zhan, S.: Chain of Thought Prompt Tuning in Vision Language Models. arXiv preprint arXiv:2304.07919 (2023)

Original

2023

Earlier work this paper cites.

Gong, Y., Ran, D., Liu, J., Wang, C., Cong, T., Wang, A., Duan, S., Wang, X.: FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts. arXiv preprint arXiv:2311.05608 (2023)

Original

2023

Earlier work this paper cites.

Han, D., Jia, X., Bai, Y., Gu, J., Liu, Y., Cao, X.: OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization. arXiv preprint arXiv:2312.04403 (2023)

Original

2023

Earlier work this paper cites.

Ji, Y., Ge, C., Kong, W., Xie, E., Liu, Z., Li, Z., Luo, P.: Large Language Models as Automated Aligners for benchmarking Vision-Language Models. arXiv preprint arXiv:2311.14580 (2023)

Original

2023

Earlier work this paper cites.

Li, J., Li, D., Savarese, S., Hoi, S.: BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In: ICML (2023)

2023

Earlier work this paper cites.

Li, L., Xie, Z., Li, M., Chen, S., Wang, P., Chen, L., Yang, Y., Wang, B., Kong, L.: Silkie: Preference Distillation for Large Visual Language Models. arXiv preprint arXiv:2312.10665 (2023)

Original

2023

Earlier work this paper cites.

Lin, B., Zhu, B., Ye, Y., Ning, M., Jin, P., Yuan, L.: Video-LLaVA: Learning United Visual Representation by Alignment Before Projection. arXiv preprint arXiv:2311.10122 (2023)

Original

2023

Earlier work this paper cites.

Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual Instruction Tuning. In: NeurIPS (2023)

2023

Earlier work this paper cites.

Liu, X., Zhu, Y., Lan, Y., Yang, C., Qiao, Y.: Query-Relevant Images Jailbreak Large Multi-Modal Models. arXiv preprint arXiv:2311.17600 (2023)

Original

2023

Cited alongside, same era.

Lyu, H., Huang, J., Zhang, D., Yu, Y., Mou, X., Pan, J., Yang, Z., Wei, Z., Luo, J.: GPT-4v(ision) as a social media analysis engine. arXiv preprint arXiv:2311.07547 (2023)

Original

2023

Cited alongside, same era.

Meta: Llama usage policy (2023), https://ai.meta.com/llama/use-policy , accessed on 10-2023

2023

Cited alongside, same era.

OpenAI: OpenAI usage policy (2023), https://openai.com/policies/usage-policies , accessed on 10-2023

2023

Cited alongside, same era.

Qi, X., Huang, K., Panda, A., Henderson, P., Wang, M., Mittal, P.: Visual Adversarial Examples Jailbreak Aligned Large Language Models. arXiv preprint arXiv:2306.13213 (2023)

Original

Zheng, L., Chiang, W.L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E.P., Zhang, H., Gonzalez, J.E., Stoica, I.: Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv preprint arXiv:2306.05685 (2023)

Original

2023

Later among the works it cites.

Zhou, H., Liu, F., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S.S., Zhou, P., Liu, J., Hua, Y., Mao, C., Wu, X., Zheng, Y., Clifton, L., Li, Z., Luo, J., Clifton, D.A.: A Survey of Large Language Models in Medicine: Progress, Application, and Challenge. arXiv preprint arXiv:2311.05112 (2023)

Original

2023

Later among the works it cites.

Zhu, B., Lin, B., Ning, M., Yan, Y., Cui, J., Wang, H., Pang, Y., Jiang, W., Zhang, J., Li, Z., et al.: LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment. arXiv preprint arXiv:2310.01852 (2023)

Original

2023

Later among the works it cites.

Cha, S., Lee, J., Lee, Y., Yang, C.: Visually Dehallucinative Instruction Generation: Know What You Don’t Know. arXiv preprint arXiv:2303.16199 (2024)

Original

2024