2024

Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks

Ying, Zonghao, Liu, Aishan, Liu, Xianglong et al.

Understand

The recent release of GPT-4o has garnered widespread attention due to its powerful general capabilities.

While its impressive performance is widely acknowledged, its safety aspects have not been sufficiently explored.
Given the potential societal impact of risky content generated by advanced generative AI such as GPT-4o, it is crucial to rigorously evaluate its safety.
In response to this question, this paper for the first time conducts a rigorous evaluation of GPT-4o against jailbreak attacks.

Built on

Perceptual-sensitive gan for generating adversarial patches
Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao · 2019
Earlier work this paper cites.
Dual attention suppression attack: Generate adversarial camouflage in physical world
Jiakai Wang, Aishan Liu, Zixin Yin, Shunchang Liu, Shiyu Tang, and Xianglong Liu · 2021
Earlier work this paper cites.
Interpreting and improving adversarial robustness of deep neural networks with neuron sensitivity
Chongzhi Zhang, Aishan Liu, Xianglong Liu, Yitao Xu, Hang Yu, Yuqing Ma, and Tianlin Li · 2021
Earlier work this paper cites.
Training a helpful and harmless assistant with reinforcement learning from human feedback
Original
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Chris Olah, Benjamin Mann, and Jared Kaplan · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe · 2022
Earlier work this paper cites.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou · 2022
Earlier work this paper cites.
Figstep: Jailbreaking large vision-language models via typographic visual prompts
Original
Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang · 2023
Earlier work this paper cites.
Llama guard: Llm-based input-output safeguard for human-ai conversations
Original
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa · 2023
Earlier work this paper cites.
Beavertails: Towards improved safety alignment of LLM via a human-preference dataset
Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang · 2023
Earlier work this paper cites.
Beavertails: Towards improved safety alignment of LLM via a human-preference dataset
Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang · 2023
Earlier work this paper cites.
G-eval: NLG evaluation using gpt-4 with better human alignment
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu · 2023
Earlier work this paper cites.

Similar

"do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models
Original
Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang · 2023
Cited alongside, same era.
Safety assessment of chinese large language models
Original
Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng, and Minlie Huang · 2023
Cited alongside, same era.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom · 2023
Cited alongside, same era.
Is chatgpt a good NLG evaluator? A preliminary study
Original
Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou · 2023
Cited alongside, same era.
Jailbroken: How does LLM safety training fail?
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt · 2023
Cited alongside, same era.
Universal and transferable adversarial attacks on aligned language models
Original
Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson · 2023
Cited alongside, same era.
Llama 2 -acceptable use policy
Meta AI · 2024
Cited alongside, same era.
Image hijacks: Adversarial images can control generative models at runtime, 2024
Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons · 2024
Cited alongside, same era.
Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks
Original
Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao · 2024
Cited alongside, same era.
Gpt-4v(ision) system card
OpenAI · 2024
Cited alongside, same era.
Large language models as moral experts? gpt-4o outperforms expert ethicist in providing moral guidance
Danica Dillion, Debanjan Mondal, Niket Tandon, and Kurt Gray
Cited in the paper.
Poisoned forgery face: Towards backdoor attacks on face forgery detection
Original
Liang et al
Cited in the paper.

Then

Chatgpt: Chat generative pre-trained transformer
OpenAI · 2024
Closest in time.
Hello gpt-4o
OpenAI · 2024
Closest in time.
Introducing gpt-4o: our fastest and most affordable flagship model
OpenAI · 2024
Closest in time.
Chatgpt android app - faq
OpenAI · 2024
Closest in time.
Usage policies
OpenAI · 2024
Closest in time.
Diagnostic performances of gpt-4o, claude 3 opus, and gemini 1.5 pro in radiology’s diagnosis please cases
Yuki Sonoda, Ryo Kurokawa, Yuta Nakamura, Jun Kanzawa, Mariko Kurokawa, Yuji Ohizumi, Wataru Gonoi, and Osamu Abe · 2024
Closest in time.
Hallucination is inevitable: An innate limitation of large language models
Original
Ziwei Xu, Sanjay Jain, and Mohan S. Kankanhalli · 2024
Closest in time.
Jailbreak vision language models via bi-modal adversarial prompt
Original
Zonghao Ying, Aishan Liu, Tianyuan Zhang, Zhengmin Yu, Siyuan Liang, Xianglong Liu, and Dacheng Tao · 2024
Closest in time.
How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge AI safety by humanizing llms
Original
Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi · 2024
Closest in time.
Openai’s gpt-4o in surgical oncology: revolutionary advances in generative artificial intelligence
Ning Zhu, Nan Zhang, Qipeng Shao, Kunming Cheng, and Haiyang Wu · 2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…