Built on

A. H. Garcia, “pyleetspeak: a python package for converting text to leetspeak,” Python Package Index (PyPI), 2017. [Online]. Available: https://pypi.org/project/pyleetspeak/
2017
Earlier work this paper cites.
D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt, “Measuring mathematical problem solving with the math dataset,” arXiv preprint arXiv:2103.03874 , 2021
Original
2021
Earlier work this paper cites.
D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, A. Jones et al. , “Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned,” arXiv preprint arXiv:2209.07858 , 2022
Original
2022
Earlier work this paper cites.
X. Zhang, C. Zhang, T. Li, Y. Huang, X. Jia, M. Hu, J. Zhang, Y. Liu, S. Ma, and C. Shen, “Jailguard: A universal detection framework for llm prompt-based attacks,” arXiv preprint arXiv:2312.10766 , 2023
Original
2023
Earlier work this paper cites.
G. Kour, M. Zalmanovici, N. Zwerdling, E. Goldbraich, O. N. Fandina, A. Anaby-Tavor, O. Raz, and E. Farchi, “Unveiling safety vulnerabilities of large language models,” arXiv preprint arXiv:2311.04124 , 2023
Original
2023
Earlier work this paper cites.
J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou, “Instruction-following evaluation for large language models,” arXiv preprint arXiv:2311.07911 , 2023
Original
2023
Earlier work this paper cites.
N. Mu, S. Chen, Z. Wang, S. Chen, D. Karamardian, L. Aljeraisy, B. Alomair, D. Hendrycks, and D. Wagner, “Can llms follow simple rules?” arXiv preprint arXiv:2311.04235 , 2023
Original
2023
Earlier work this paper cites.
T. Chang, J. Wiens, T. Schnabel, and A. Swaminathan, “Measuring steerability in large language models,” in Neurips Safe Generative AI Workshop 2024 , 2024
2024
Earlier work this paper cites.

Similar

W. Luo, S. Ma, X. Liu, X. Guo, and C. Xiao, “Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks,” 2024
2024
Cited alongside, same era.
D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman, “GPQA: A graduate-level google-proof q&a benchmark,” arXiv preprint arXiv:2311.12022 , 2024
Original
2024
Cited alongside, same era.
Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, T. Li, M. Ku, K. Wang, A. Zhuang, R. Fan, X. Yue, and W. Chen, “Mmlu-pro: A more robust and challenging multi-task language understanding benchmark,” arXiv preprint arXiv:2406.01574 , 2024
Original
2024
Cited alongside, same era.
E. Miehling, M. Desmond, K. N. Ramamurthy, E. M. Daly, P. Dognin, J. Rios, D. Bouneffouf, and M. Liu, “Evaluating the prompt steerability of large language models,” arXiv preprint arXiv:2411.12405 , 2024
Original
2024
Cited alongside, same era.

Then

“security_steerability,” Hugging Face Dataset Hub, 2025, accessed: 2025-04-23. [Online]. Available: https://huggingface.co/datasets/itayhf/security_steerability
2025
Closest in time.
K. Chen, Z. He, T. Shi, and K. Lerman, “Steer-bench: A benchmark for evaluating the steerability of large language models,” 2025. [Online]. Available: https://arxiv.org/abs/2505.20645
Original
2025
Closest in time.
I. Joshi, S. Shahid, S. M. Venneti, M. Vasu, Y. Zheng, Y. Li, B. Krishnamurthy, and G. Y.-Y. Chan, “Coprompter: User-centric evaluation of llm instruction alignment for improved prompt engineering,” in Proceedings of the 30th International Conference on Intelligent User Interfaces , 2025, pp. 341–365
2025
Closest in time.
OpenAI, “O3 mini system card,” https://cdn.openai.com/o3-mini-system-card-feb10.pdf , 2025, accessed: 2025-04-23
2025
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2025

Security Steerability is All You Need

Hazan, Itay, Habler, Idan, Bitton, Ron et al.

Understand

The adoption of Generative AI (GenAI) in applications inevitably comes with the expansion of the attack surface, combining new security threats along with the traditional ones.

Consequently, numerous research and industrial initiatives aim to mitigate the GenAI related security threats by developing evaluation methods and designing defenses.
However, while most of the GenAI security work focuses on universal threats (e.g.
'How to build a bomb'), there is significantly less discussion on application-level security and how to evaluate and mitigate it.

Built on

A. H. Garcia, “pyleetspeak: a python package for converting text to leetspeak,” Python Package Index (PyPI), 2017. [Online]. Available: https://pypi.org/project/pyleetspeak/
2017
Earlier work this paper cites.
D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt, “Measuring mathematical problem solving with the math dataset,” arXiv preprint arXiv:2103.03874 , 2021
Original
2021
Earlier work this paper cites.
D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, A. Jones et al. , “Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned,” arXiv preprint arXiv:2209.07858 , 2022
Original
2022
Earlier work this paper cites.
X. Zhang, C. Zhang, T. Li, Y. Huang, X. Jia, M. Hu, J. Zhang, Y. Liu, S. Ma, and C. Shen, “Jailguard: A universal detection framework for llm prompt-based attacks,” arXiv preprint arXiv:2312.10766 , 2023
Original
2023
Earlier work this paper cites.
G. Kour, M. Zalmanovici, N. Zwerdling, E. Goldbraich, O. N. Fandina, A. Anaby-Tavor, O. Raz, and E. Farchi, “Unveiling safety vulnerabilities of large language models,” arXiv preprint arXiv:2311.04124 , 2023
Original
2023
Earlier work this paper cites.
J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou, “Instruction-following evaluation for large language models,” arXiv preprint arXiv:2311.07911 , 2023
Original
2023
Earlier work this paper cites.
N. Mu, S. Chen, Z. Wang, S. Chen, D. Karamardian, L. Aljeraisy, B. Alomair, D. Hendrycks, and D. Wagner, “Can llms follow simple rules?” arXiv preprint arXiv:2311.04235 , 2023
Original
2023
Earlier work this paper cites.
T. Chang, J. Wiens, T. Schnabel, and A. Swaminathan, “Measuring steerability in large language models,” in Neurips Safe Generative AI Workshop 2024 , 2024
2024
Earlier work this paper cites.

Similar

W. Luo, S. Ma, X. Liu, X. Guo, and C. Xiao, “Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks,” 2024
2024
Cited alongside, same era.
D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman, “GPQA: A graduate-level google-proof q&a benchmark,” arXiv preprint arXiv:2311.12022 , 2024
Original
2024
Cited alongside, same era.
Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, T. Li, M. Ku, K. Wang, A. Zhuang, R. Fan, X. Yue, and W. Chen, “Mmlu-pro: A more robust and challenging multi-task language understanding benchmark,” arXiv preprint arXiv:2406.01574 , 2024
Original
2024
Cited alongside, same era.
E. Miehling, M. Desmond, K. N. Ramamurthy, E. M. Daly, P. Dognin, J. Rios, D. Bouneffouf, and M. Liu, “Evaluating the prompt steerability of large language models,” arXiv preprint arXiv:2411.12405 , 2024
Original
2024
Cited alongside, same era.

Then

“security_steerability,” Hugging Face Dataset Hub, 2025, accessed: 2025-04-23. [Online]. Available: https://huggingface.co/datasets/itayhf/security_steerability
2025
Closest in time.
K. Chen, Z. He, T. Shi, and K. Lerman, “Steer-bench: A benchmark for evaluating the steerability of large language models,” 2025. [Online]. Available: https://arxiv.org/abs/2505.20645
Original
2025
Closest in time.
I. Joshi, S. Shahid, S. M. Venneti, M. Vasu, Y. Zheng, Y. Li, B. Krishnamurthy, and G. Y.-Y. Chan, “Coprompter: User-centric evaluation of llm instruction alignment for improved prompt engineering,” in Proceedings of the 30th International Conference on Intelligent User Interfaces , 2025, pp. 341–365
2025
Closest in time.
OpenAI, “O3 mini system card,” https://cdn.openai.com/o3-mini-system-card-feb10.pdf , 2025, accessed: 2025-04-23
2025
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

A. Souly, Q. Lu, D. Bowen, T. Trinh, E. Hsieh, S. Pandey, P. Abbeel, J. Svegliato, S. Emmons, O. Watkins, and S. Toyer, “A strongreject for empty jailbreaks,” arXiv preprint arXiv:2402.10260 , 2024

Original

2024

Cited alongside, same era.

S. Wan, C. Nikolaidis, D. Song, D. Molnar, J. Crnkovich, J. Grace, M. Bhatt, S. Chennabasappa, S. Whitman, S. Ding, V. Ionescu, Y. Li, and J. Saxe, “Cyberseceval 3: Advancing the evaluation of cybersecurity risks and capabilities in large language models,” arXiv preprint arXiv:2408.01605 , 2024

Original

2024

Cited alongside, same era.

P. Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V. Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Tramer, H. Hassani, and E. Wong, “Jailbreakbench: An open robustness benchmark for jailbreaking large language models,” arXiv preprint arXiv:2404.01318 , 2024

Original

2024

Cited alongside, same era.

M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, D. Forsyth, and D. Hendrycks, “Harmbench: A standardized evaluation framework for automated red teaming and robust refusal,” arXiv preprint arXiv:2402.04249 , 2024

Original

2024

Cited alongside, same era.

OWASP, “Owasp top 10 for llm applications 2025,” https://genai.owasp.org/resource/owasp-top-10-for-llm-applications-2025/, 2024, accessed: 2025-04-23

2025

Closest in time.

——, “Agentic ai threats and mitigations,” https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/, 2025, accessed: 2025-04-23

2025

Closest in time.

F. S. Marcondes, A. Gala, R. Magalhães, F. P. de Britto, D. Durães, and P. Novais, “Using ollama,” in Natural Language Analytics with Generative Large-Language Models: A Practical Approach with Ollama and Open-Source LLMs . Springer, 2025, pp. 23–35

2025

Closest in time.