Built on

A study on data augmentation of reverberant speech for robust speech recognition
Tom Ko, Vijayaditya Peddinti, Daniel Povey, Michael L Seltzer, and Sanjeev Khudanpur · 2017
Earlier work this paper cites.
A survey on in-context learning
Original
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui · 2022
Earlier work this paper cites.
Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models
Original
Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou · 2023
Earlier work this paper cites.
Multilingual jailbreak challenges in large language models
Original
Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing · 2023
Earlier work this paper cites.
Pengi: An audio language model for audio tasks
Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang · 2023
Earlier work this paper cites.
Llama guard: Llm-based input-output safeguard for human-ai conversations
Original
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al · 2023
Earlier work this paper cites.
Robust speech recognition via large-scale weak supervision
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever · 2023
Earlier work this paper cites.
Salmonn: Towards generic hearing abilities for large language models
Original
Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang · 2023
Earlier work this paper cites.
Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities
Original
Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu · 2023
Earlier work this paper cites.
Universal and transferable adversarial attacks on aligned language models, 2023
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson · 2023
Earlier work this paper cites.
Voicebench: Benchmarking llm-based voice assistants
Original
Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T Tan, and Haizhou Li · 2024
Earlier work this paper cites.
Qwen2-audio technical report
Original
Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al · 2024
Earlier work this paper cites.
Ichigo: Mixed-modal early-fusion realtime voice assistant
Original
Alan Dao, Dinh Bach Vu, and Huy Hoang Ha · 2024
Earlier work this paper cites.

Similar

Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities
Original
Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha · 2024
Cited alongside, same era.
Meralion-audiollm: Technical report
Original
Yingxu He, Zhuohan Liu, Shuo Sun, Bin Wang, Wenyu Zhang, Xunlong Zou, Nancy F Chen, and Ai Ti Aw · 2024
Cited alongside, same era.
Distilling an end-to-end voice assistant without instruction training data
Original
William Held, Ella Li, Michael Ryan, Weiyan Shi, Yanzhe Zhang, and Diyi Yang · 2024
Cited alongside, same era.
Plentiful jailbreaks with string compositions
Original
Brian RY Huang · 2024
Cited alongside, same era.
Best-of-n jailbreaking

Then

Distract large language models for automatic jailbreak attack
Original
Zeguan Xiao, Yan Yang, Guanhua Chen, and Yun Chen · 2024
Later among the works it cites.
Minicpm-v: A gpt-4v level mllm on your phone
Original
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al · 2024
Later among the works it cites.
Unveiling the safety of gpt-4o: An empirical study using jailbreak attacks
Original
Zonghao Ying, Aishan Liu, Xianglong Liu, and Dacheng Tao · 2024
Later among the works it cites.
Weak-to-strong jailbreaking on large language models
Original
Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu-Xiang Wang, and William Yang Wang · 2024
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2025

Multilingual and Multi-Accent Jailbreaking of Audio LLMs

Roh, Jaechul, Shejwalkar, Virat, Houmansadr, Amir

Understand

Large Audio Language Models (LALMs) have significantly advanced audio understanding but introduce critical security risks, particularly through audio jailbreaks.

While prior work has focused on English-centric attacks, we expose a far more severe vulnerability: adversarial multilingual and multi-accent audio jailbreaks, where linguistic and acoustic variations dramatically amplify attack success.
In this paper, we introduce Multi-AudioJail, the first systematic framework to exploit these vulnerabilities through (1) a novel dataset of adversarially perturbed multilingual/multi-accent audio jailbreaking prompts, and (2) a hierarchical evaluation pipeline revealing that how acoustic perturbations (e.g., reverberation, echo, and whisper effects) interacts with cross-lingual phonetics to cause jailbreak success rates (JSRs) to surge by up to +57.25 percentage points (e.g., reverberated Kenyan-accented attack on MERaLiON).
Crucially, our work further reveals that multimodal LLMs are inherently more vulnerable than unimodal systems: attackers need only exploit the weakest link (e.g., non-English audio inputs) to compromise the entire model, which we empirically show by multilingual audio-only attacks achieving 3.1x higher success rates than text-only attacks.

Built on

A study on data augmentation of reverberant speech for robust speech recognition
Tom Ko, Vijayaditya Peddinti, Daniel Povey, Michael L Seltzer, and Sanjeev Khudanpur · 2017
Earlier work this paper cites.
A survey on in-context learning
Original
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui · 2022
Earlier work this paper cites.
Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models
Original
Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou · 2023
Earlier work this paper cites.
Multilingual jailbreak challenges in large language models
Original
Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing · 2023
Earlier work this paper cites.
Pengi: An audio language model for audio tasks
Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang · 2023
Earlier work this paper cites.
Llama guard: Llm-based input-output safeguard for human-ai conversations
Original
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al · 2023
Earlier work this paper cites.
Robust speech recognition via large-scale weak supervision
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever · 2023
Earlier work this paper cites.
Salmonn: Towards generic hearing abilities for large language models
Original
Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang · 2023
Earlier work this paper cites.
Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities
Original
Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu · 2023
Earlier work this paper cites.
Universal and transferable adversarial attacks on aligned language models, 2023
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson · 2023
Earlier work this paper cites.
Voicebench: Benchmarking llm-based voice assistants
Original
Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T Tan, and Haizhou Li · 2024
Earlier work this paper cites.
Qwen2-audio technical report
Original
Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al · 2024
Earlier work this paper cites.
Ichigo: Mixed-modal early-fusion realtime voice assistant
Original
Alan Dao, Dinh Bach Vu, and Huy Hoang Ha · 2024
Earlier work this paper cites.

Similar

Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities
Original
Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha · 2024
Cited alongside, same era.
Meralion-audiollm: Technical report
Original
Yingxu He, Zhuohan Liu, Shuo Sun, Bin Wang, Wenyu Zhang, Xunlong Zou, Nancy F Chen, and Ai Ti Aw · 2024
Cited alongside, same era.
Distilling an end-to-end voice assistant without instruction training data
Original
William Held, Ella Li, Michael Ryan, Weiyan Shi, Yanzhe Zhang, and Diyi Yang · 2024
Cited alongside, same era.
Plentiful jailbreaks with string compositions
Original
Brian RY Huang · 2024
Cited alongside, same era.
Best-of-n jailbreaking

Then

Distract large language models for automatic jailbreak attack
Original
Zeguan Xiao, Yan Yang, Guanhua Chen, and Yun Chen · 2024
Later among the works it cites.
Minicpm-v: A gpt-4v level mllm on your phone
Original
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al · 2024
Later among the works it cites.
Unveiling the safety of gpt-4o: An empirical study using jailbreak attacks
Original
Zonghao Ying, Aishan Liu, Xianglong Liu, and Dacheng Tao · 2024
Later among the works it cites.
Weak-to-strong jailbreaking on large language models
Original
Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu-Xiang Wang, and William Yang Wang · 2024
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

John Hughes, Sara Price, Aengus Lynch, Rylan Schaeffer, Fazl Barez, Sanmi Koyejo, Henry Sleight, Erik Jones, Ethan Perez, and Mrinank Sharma · 2024

Cited alongside, same era.

Artprompt: Ascii art-based jailbreak attacks against aligned llms

Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran · 2024

Cited alongside, same era.

Attackeval: How to evaluate the effectiveness of jailbreak attacking on large language models

Original

Mingyu Jin, Suiyuan Zhu, Beichen Wang, Zihao Zhou, Chong Zhang, Yongfeng Zhang, et al · 2024

Cited alongside, same era.

Advwave: Stealthy adversarial jailbreak attack against large audio-language models

Original

Mintong Kang, Chejian Xu, and Bo Li · 2024

Cited alongside, same era.

A cross-language investigation into jailbreak attacks in large language models.(2024), 2024b

Jie Li, Yi Liu, Chongyang Liu, Ling Shi, Xiaoning Ren, Yaowen Zheng, Yang Liu, and Yinxing Xue · 2024

Cited alongside, same era.

The llama 3 herd of models, 2024

Original

AI @ Meta Llama Team · 2024

Cited alongside, same era.

Jailbreakeval: An integrated toolkit for evaluating jailbreak attempts against large language models

Original

Delong Ran, Jinyuan Liu, Yichen Gong, Jingyi Zheng, Xinlei He, Tianshuo Cong, and Anyu Wang · 2024

Cited alongside, same era.

A strongreject for empty jailbreaks

Original

Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, et al · 2024

Cited alongside, same era.

Do as i say not as i do’: A semi-automated approach for jailbreak prompt attack against multimodal llms

Chun Wai Chiu, Linghan Huang, Bo Li, and Huaming Chen · 2025

Closest in time.

Ultravox v0.4.1 llama 3 8b

Fixie.ai · 2025

Closest in time.

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

Original

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al · 2025

Closest in time.

” i am bad”: Interpreting stealthy, universal and robust audio jailbreaks in audio-language models