Built on

D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, et al
Original
2022
Earlier work this paper cites.
Z. Zhang, L. Lei, L. Wu, R. Sun, Y. Huang, C. Long, X. Liu, X. Lei, J. Tang, and M. Huang, “Safetybench: Evaluating the safety of large language models with multiple choice questions,” arXiv preprint arXiv:2309.07045
Original
2023
Earlier work this paper cites.
B. Vidgen, N. Scherrer, H. R. Kirk, R. Qian, A. Kannappan, S. A. Hale, and P. Röttger, “Simplesafetytests: a test suite for identifying critical safety risks in large language models,” arXiv preprint arXiv:2311.08370
Original
2023
Earlier work this paper cites.
A. Biswas and W. Talukdar, “Guardrails for trust, safety, and ethical development and deployment of large language models (llm),” Journal of Science & Technology
2023
Earlier work this paper cites.
H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, et al
Original
2023
Earlier work this paper cites.
Y. Huang, S. Gupta, M. Xia, K. Li, and D. Chen, “Catastrophic jailbreak of open-source llms via exploiting generation,” arXiv preprint arXiv:2310.06987
Original
2023
Earlier work this paper cites.
A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” arXiv preprint arXiv:2307.15043
Original
2023
Earlier work this paper cites.
X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “" do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” arXiv preprint arXiv:2308.03825
Original
2023
Earlier work this paper cites.
T. Xie, X. Qi, Y. Zeng, Y. Huang, U. M. Sehwag, K. Huang, L. He, B. Wei, D. Li, Y. Sheng, et al
Original
2024
Earlier work this paper cites.

Similar

X. Yuan, J. Li, D. Wang, Y. Chen, X. Mao, L. Huang, H. Xue, W. Wang, K. Ren, and J. Wang, “S-eval: Automatic and adaptive test generation for benchmarking safety evaluation of large language models,” arXiv preprint arXiv:2405.14191
Original
2024
Cited alongside, same era.
W. Zhang, X. Lei, Z. Liu, M. An, B. Yang, K. Zhao, K. Wang, and S. Lian, “Chisafetybench: A chinese hierarchical safety benchmark for large language models,” arXiv preprint arXiv:2406.10311
Original
2024
Cited alongside, same era.
Z. Ying, A. Liu, S. Liang, L. Huang, J. Guo, W. Zhou, X. Liu, and D. Tao, “Safebench: A safety evaluation framework for multimodal large language models,” arXiv preprint arXiv:2410.18927
Original
2024
Cited alongside, same era.
M. Huang, X. Liu, S. Zhou, M. Zhang, C. Tan, P. Wang, Q. Guo, Z. Xu, L. Li, Z. Lei, et al
Original
2024
Cited alongside, same era.

Then

Z. Zhang, Y. Lu, J. Ma, D. Zhang, R. Li, P. Ke, H. Sun, L. Sha, Z. Sui, H. Wang, et al
Original
2024
Later among the works it cites.
A. Souly, Q. Lu, D. Bowen, T. Trinh, E. Hsieh, S. Pandey, P. Abbeel, J. Svegliato, S. Emmons, O. Watkins, et al
Original
2024
Later among the works it cites.
M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, et al
Original
2024
Later among the works it cites.
A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does llm safety training fail?,” Advances in Neural Information Processing Systems
2024
Later among the works it cites.
P. Gupta, L. Q. Yau, H. H. Low, I. Lee, H. M. Lim, Y. X. Teoh, J. H. Koh, D. W. Liew, R. Bhardwaj, R. Bhardwaj, et al
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2025

o3-mini vs DeepSeek-R1: Which One is Safer?

Arrieta, Aitor, Ugarte, Miriam, Valle, Pablo et al.

Understand

The irruption of DeepSeek-R1 constitutes a turning point for the AI industry in general and the LLMs in particular.

Its capabilities have demonstrated outstanding performance in several tasks, including creative thinking, code generation, maths and automated program repair, at apparently lower execution cost.
However, LLMs must adhere to an important qualitative property, i.e., their alignment with safety and human values.
A clear competitor of DeepSeek-R1 is its American counterpart, OpenAI's o3-mini model, which is expected to set high standards in terms of performance, safety and cost.

Built on

D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, et al
Original
2022
Earlier work this paper cites.
Z. Zhang, L. Lei, L. Wu, R. Sun, Y. Huang, C. Long, X. Liu, X. Lei, J. Tang, and M. Huang, “Safetybench: Evaluating the safety of large language models with multiple choice questions,” arXiv preprint arXiv:2309.07045
Original
2023
Earlier work this paper cites.
B. Vidgen, N. Scherrer, H. R. Kirk, R. Qian, A. Kannappan, S. A. Hale, and P. Röttger, “Simplesafetytests: a test suite for identifying critical safety risks in large language models,” arXiv preprint arXiv:2311.08370
Original
2023
Earlier work this paper cites.
A. Biswas and W. Talukdar, “Guardrails for trust, safety, and ethical development and deployment of large language models (llm),” Journal of Science & Technology
2023
Earlier work this paper cites.
H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, et al
Original
2023
Earlier work this paper cites.
Y. Huang, S. Gupta, M. Xia, K. Li, and D. Chen, “Catastrophic jailbreak of open-source llms via exploiting generation,” arXiv preprint arXiv:2310.06987
Original
2023
Earlier work this paper cites.
A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” arXiv preprint arXiv:2307.15043
Original
2023
Earlier work this paper cites.
X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “" do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” arXiv preprint arXiv:2308.03825
Original
2023
Earlier work this paper cites.
T. Xie, X. Qi, Y. Zeng, Y. Huang, U. M. Sehwag, K. Huang, L. He, B. Wei, D. Li, Y. Sheng, et al
Original
2024
Earlier work this paper cites.

Similar

X. Yuan, J. Li, D. Wang, Y. Chen, X. Mao, L. Huang, H. Xue, W. Wang, K. Ren, and J. Wang, “S-eval: Automatic and adaptive test generation for benchmarking safety evaluation of large language models,” arXiv preprint arXiv:2405.14191
Original
2024
Cited alongside, same era.
W. Zhang, X. Lei, Z. Liu, M. An, B. Yang, K. Zhao, K. Wang, and S. Lian, “Chisafetybench: A chinese hierarchical safety benchmark for large language models,” arXiv preprint arXiv:2406.10311
Original
2024
Cited alongside, same era.
Z. Ying, A. Liu, S. Liang, L. Huang, J. Guo, W. Zhou, X. Liu, and D. Tao, “Safebench: A safety evaluation framework for multimodal large language models,” arXiv preprint arXiv:2410.18927
Original
2024
Cited alongside, same era.
M. Huang, X. Liu, S. Zhou, M. Zhang, C. Tan, P. Wang, Q. Guo, Z. Xu, L. Li, Z. Lei, et al
Original
2024
Cited alongside, same era.

Then

Z. Zhang, Y. Lu, J. Ma, D. Zhang, R. Li, P. Ke, H. Sun, L. Sha, Z. Sui, H. Wang, et al
Original
2024
Later among the works it cites.
A. Souly, Q. Lu, D. Bowen, T. Trinh, E. Hsieh, S. Pandey, P. Abbeel, J. Svegliato, S. Emmons, O. Watkins, et al
Original
2024
Later among the works it cites.
M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, et al
Original
2024
Later among the works it cites.
A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does llm safety training fail?,” Advances in Neural Information Processing Systems
2024
Later among the works it cites.
P. Gupta, L. Q. Yau, H. H. Low, I. Lee, H. M. Lim, Y. X. Teoh, J. H. Koh, D. W. Liew, R. Bhardwaj, R. Bhardwaj, et al
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

L. Li, B. Dong, R. Wang, X. Hu, W. Zuo, D. Lin, Y. Qiao, and J. Shao, “Salad-bench: A hierarchical and comprehensive safety benchmark for large language models,” arXiv preprint arXiv:2402.05044

Original

2024

Cited alongside, same era.

J. Ji, M. Liu, J. Dai, X. Pan, C. Zhang, C. Bian, B. Chen, R. Sun, Y. Wang, and Y. Yang, “Beavertails: Towards improved safety alignment of LLM via a human-preference dataset,” Advances in Neural Information Processing Systems

2024

Cited alongside, same era.

[Online]

“European Commission AI Act.” https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai , 2024 · 2024

Cited alongside, same era.

[Online]

“Artificial Intelligence Act (Regulation (EU) 2024/1689), Official Journal version of 13 June 2024.” https://eur-lex.europa.eu/eli/reg/2024/1689/oj , 2024 · 2024

Cited alongside, same era.

2024

Later among the works it cites.

D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al

Original

2025

Closest in time.

M. Ugarte, P. Valle, J. A. Parejo, S. Segura, and A. Arrieta, “Astral: Automated safety testing of large language models,” in 2025 IEEE/ACM International Conference on Automation of Software Test (AST)

2025

Closest in time.

A. Arrieta, M. Ugarte, P. Valle, J. A. Parejo, and S. Segura, “Early external safety testing of openai’s o3-mini: Insights from the pre-deployment evaluation,” 2025

2025

Closest in time.