Fetching the paper…
Reading the bibliography…
Multimodal large language models (MLLMs) excel in vision-language tasks but also pose significant risks of generating harmful content, particularly through jailbreak attacks.
A baseline for detecting misclassified and out-of-distribution examples in neural networks
Dan Hendrycks and Kevin Gimpel · 2016
Earlier work this paper cites.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Earlier work this paper cites.
Test-time prompt tuning for zero-shot generalization in vision-language models
Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao · 2022
Earlier work this paper cites.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Earlier work this paper cites.
Autoencoders
Dor Bank, Noam Koenigstein, and Raja Giryes · 2023
Earlier work this paper cites.
Improving image captioning descriptiveness by ranking and llm-based fusion
Simone Bianco, Luigi Celona, Marco Donzella, and Paolo Napoletano · 2023
Earlier work this paper cites.
Diverse data augmentation with diffusions for effective test-time prompt tuning
Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo · 2023
Earlier work this paper cites.
Figstep: Jailbreaking large vision-language models via typographic visual prompts, 2023
Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang · 2023
Earlier work this paper cites.
Llama guard: Llm-based input-output safeguard for human-ai conversations
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al · 2023
Earlier work this paper cites.
Visual instruction tuning, 2023
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee · 2023
Earlier work this paper cites.
Swapprompt: test-time prompt adaptation for vision-language models
Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu · 2023
Earlier work this paper cites.
Label-free concept bottleneck models
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui-Wei Weng · 2023
Earlier work this paper cites.
Prompting large language models with answer heuristics for knowledge-based visual question answering
Zhenwei Shao, Zhou Yu, Meng Wang, and Jun Yu · 2023
Earlier work this paper cites.
Cogvlm: Visual expert for pretrained language models, 2023
Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang · 2023
Cited alongside, same era.
Temporal coherent test-time optimization for robust video classification
Chenyu Yi, Siyuan Yang, Yufei Wang, Haoliang Li, Yap-Peng Tan, and Alex C Kot · 2023
Cited alongside, same era.
Mm-vet: Evaluating large multimodal models for integrated capabilities
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang · 2023
Cited alongside, same era.
Mllmguard: A multi-dimensional safety evaluation suite for multimodal large language models
Tianle Gu, Zeyang Zhou, Kexin Huang, Liang Dandan, Yixu Wang, Haiquan Zhao, Yuanqi Yao, Yujiu Yang, Yan Teng, Yu Qiao, et al · 2024
Cited alongside, same era.
Llavaguard: Vlm-based safeguards for vision dataset curation and safety assessment
Personalized multimodal large language models: A survey
Junda Wu, Hanjia Lyu, Yu Xia, Zhehao Zhang, Joe Barrow, Ishita Kumar, Mehrnoosh Mirtaheri, Hongjie Chen, Ryan A Rossi, Franck Dernoncourt, et al · 2024
Later among the works it cites.
V?: Guided visual search as a core mechanism in multimodal llms
Penghao Wu and Saining Xie · 2024
Later among the works it cites.
Efficient adversarial training in llms with continuous attacks
Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, and Leo Schwinn · 2024
Later among the works it cites.
Gradsafe: Detecting jailbreak prompts for llms via safety-critical gradient analysis
Yueqi Xie, Minghong Fang, Renjie Pi, and Neil Gong · 2024
Later among the works it cites.
Jailbreak attacks and defenses against large language models: A survey
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Lukas Helff, Felix Friedrich, Manuel Brack, Kristian Kersting, and Patrick Schramowski · 2024
Cited alongside, same era.
Effective and efficient adversarial detection for vision-language models via a single vector
Youcheng Huang, Fengbin Zhu, Jingkun Tang, Pan Zhou, Wenqiang Lei, Jiancheng Lv, and Tat-Seng Chua · 2024
Cited alongside, same era.
Efficient test-time adaptation of vision-language models
Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik, and Eric Xing · 2024
Cited alongside, same era.
Injecguard: Benchmarking and mitigating over-defense in prompt injection guardrail models
Hao Li and Xiaogeng Liu · 2024
Cited alongside, same era.
Exploring diverse methods in visual question answering
Panfeng Li, Qikai Yang, Xieming Geng, Wenjing Zhou, Zhicheng Ding, and Yi Nian · 2024
Cited alongside, same era.
Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks, 2024
Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao · 2024
Cited alongside, same era.
Designprompt: Using multimodal interaction for design exploration with generative ai
Xiaohan Peng, Janin Koch, and Wendy E Mackay · 2024
Cited alongside, same era.
Metaood: Automatic selection of ood detection models
Yuehan Qin, Yichi Zhang, Yi Nian, Xueying Ding, and Yue Zhao · 2024
Cited alongside, same era.
Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li · 2024
Later among the works it cites.
Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al · 2024
Later among the works it cites.
Safety fine-tuning at (almost) no cost: A baseline for vision large language models
Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales · 2024
Later among the works it cites.
Yilei Jiang, Xinyan Gao, Tianshuo Peng, Yingshui Tan, Xiaoyong Zhu, Bo Zheng, and Xiangyu Yue · 2025
Closest in time.
Secure on-device video ood detection without backpropagation
Li Li, Peilin Cai, Yuxiao Zhou, Zhiyu Ni, Renjie Liang, You Qin, Yi Nian, Zhengzhong Tu, Xiyang Hu, and Yue Zhao · 2025
Closest in time.
Huawei Lin, Yingjie Lao, Tong Geng, Tan Yu, and Weijie Zhao · 2025
Closest in time.
Can multimodal llms perform time series anomaly detection?
Xiongxiao Xu, Haoran Wang, Yueqing Liang, Philip S Yu, Yue Zhao, and Kai Shu · 2025
Closest in time.
Jailbreaking multimodal large language models via shuffle inconsistency
Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Jialing Tao, YueFeng Chen, Hui Xue, and Xingxing Wei · 2025
Closest in time.