Fetching the paper…
Reading the bibliography…
The rapid advancement of large language models (LLMs) introduces dual-use capabilities that could both threaten and bolster national security and public safety (NSPS).
Universal adversarial triggers for attacking and analyzing NLP
E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh · 2019
Earlier work this paper cites.
AutoPrompt: Eliciting knowledge from language models with automatically generated prompts
T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh · 2020
Earlier work this paper cites.
Adversarially constructed evaluation sets are more challenging, but may not be fair, 2021
J. Phang, A. Chen, W. Huang, and S. R. Bowman · 2021
Earlier work this paper cites.
Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned, 2022
D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, A. Jones, S. Bowman, A. Chen, T. Conerly, N. DasSarma, D. Drain, N. Elhage, S. El-Showk, S. Fort, Z. Hatfield-Dodds, T. Henighan, D. Hernandez, T. Hume, J. Jacobson, S. Johnston, S. Kravec, C. Olsson, S. Ringer, E. Tran-Johnson, D. Amodei, T. Brown, N. Joseph, S. McCandlish, C. Olah, J. Kaplan, and J. Clark · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback, 2022
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe · 2022
Earlier work this paper cites.
Bbq: A hand-built bias benchmark for question answering, 2022
A. Parrish, A. Chen, N. Nangia, V. Padmakumar, J. Phang, J. Thompson, P. M. Htut, and S. R. Bowman · 2022
Earlier work this paper cites.
Red teaming language models with language models
E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving · 2022
Earlier work this paper cites.
Frontier ai regulation: Managing emerging risks to public safety
M. Anderljung, J. Barnhart, A. Korinek, J. Leung, C. O’Keefe, J. Whittlestone, S. Avin, M. Brundage, J. Bullock, D. Cass-Beggs, et al · 2023
Earlier work this paper cites.
Explore, establish, exploit: Red teaming language models from scratch
S. Casper, J. Lin, J. Kwon, G. Culp, and D. Hadfield-Menell · 2023
Earlier work this paper cites.
Jailbreaking black box large language models in twenty queries
P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong · 2023
Earlier work this paper cites.
P. Ding, J. Kuang, D. Ma, X. Cao, Y. Xian, J. Chen, and S. Huang · 2023
Earlier work this paper cites.
Mart: Improving llm safety with multi-round automatic red-teaming
S. Ge, C. Zhou, R. Hou, M. Khabsa, Y.-C. Wang, Q. Wang, J. Han, and Y. Mao · 2023
Earlier work this paper cites.
An overview of catastrophic ai risks, 2023
D. Hendrycks, M. Mazeika, and T. Woodside · 2023
Earlier work this paper cites.
ToxicChat: Unveiling hidden challenges of toxicity detection in real-world user-AI conversation
Z. Lin, Z. Wang, Y. Tong, Y. Wang, Y. Guo, Y. Wang, and J. Shang · 2023
Earlier work this paper cites.
Tdc 2023 (llm edition): The trojan detection challenge
M. Mazeika, A. Zou, N. Mu, L. Phan, Z. Wang, C. Yu, A. Khoja, F. Jiang, A. O’Gara, E. Sakhaee, Z. Xiang, A. Rajabi, D. Hendrycks, R. Poovendran, B. Li, and D. Forsyth · 2023
Earlier work this paper cites.
Tree of attacks: Jailbreaking black-box llms automatically
A. Mehrotra, M. Zampetakis, P. Kassianik, B. Nelson, H. Anderson, Y. Singer, and A. Karbasi · 2023
Earlier work this paper cites.
Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts, 2023
J. Yu, X. Lin, Z. Yu, and X. Xing · 2023
Earlier work this paper cites.
Automatic pseudo-harmful prompt generation for evaluating false refusals in large language models
B. An, S. Zhu, R. Zhang, M.-A. Panaitescu-Liess, Y. Xu, and F. Huang · 2024
Earlier work this paper cites.
Many-shot jailbreaking
C. Anil, E. Durmus, M. Sharma, J. Benton, S. Kundu, J. Batson, N. Rimsky, M. Tong, J. Mu, D. Ford, et al · 2024
Earlier work this paper cites.
Sabotage evaluations for frontier models, 2024
J. Benton, M. Wagner, E. Christiansen, C. Anil, E. Perez, J. Srivastav, E. Durmus, D. Ganguli, S. Kravec, B. Shlegeris, J. Kaplan, H. Karnofsky, E. Hubinger, R. Grosse, S. R. Bowman, and D. Duvenaud · 2024
Earlier work this paper cites.
Multilingual jailbreak challenges in large language models
Y. Deng, W. Zhang, S. J. Pan, and L. Bing · 2024
Earlier work this paper cites.
Attacking large language models with projected gradient descent
S. Geisler, T. Wollschläger, M. Abdalla, J. Gasteiger, and S. Günnemann · 2024
Earlier work this paper cites.
Refusal-trained llms are easily jailbroken as browser agents, 2024
P. Kumar, E. Lau, S. Vijayakumar, T. Trinh, S. R. Team, E. Chang, V. Robinson, S. Hendryx, S. Zhou, M. Fredrikson, S. Yue, and Z. Wang · 2024
Cited alongside, same era.
PRP: Propagating universal perturbations to attack large language model guard-rails
N. Mangaokar, A. Hooda, J. Choi, S. Chandrashekaran, K. Fawaz, S. Jha, and A. Prakash · 2024
Cited alongside, same era.
Generative ai misuse: A taxonomy of tactics and insights from real-world data, 2024
N. Marchal, R. Xu, R. Elasmar, I. Gabriel, B. Goldberg, and W. Isaac · 2024
Cited alongside, same era.
Harmbench: A standardized evaluation framework for automated red teaming and robust refusal, 2024
M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, D. Forsyth, and D. Hendrycks · 2024
Cited alongside, same era.
Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts, 2024
J. Yu, X. Lin, Z. Yu, and X. Xing · 2024
Later among the works it cites.
Refuse whenever you feel unsafe: Improving safety in llms via decoupled refusal training, 2024
Y. Yuan, W. Jiao, W. Wang, J. tse Huang, J. Xu, T. Liang, P. He, and Z. Tu · 2024
Later among the works it cites.
Ai risk categorization decoded (air 2024): From government regulations to corporate policies, 2024b
Y. Zeng, K. Klyman, A. Zhou, Y. Yang, M. Pan, R. Jia, D. Song, P. Liang, and B. Li · 2024
Later among the works it cites.
Robust prompt optimization for defending language models against jailbreaking attacks, 2024
A. Zhou, B. Li, and H. Wang · 2024
Later among the works it cites.
Improving alignment and robustness with circuit breakers, 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Y. Mou, S. Zhang, and W. Ye · 2024
Cited alongside, same era.
Fine-tuning aligned language models compromises safety, even when users do not intend to!
X. Qi, Y. Zeng, T. Xie, P.-Y. Chen, R. Jia, P. Mittal, and P. Henderson · 2024
Cited alongside, same era.
Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues, 2024
Q. Ren, H. Li, D. Liu, Z. Xie, X. Lu, Y. Qiao, L. Sha, J. Yan, L. Ma, and J. Shao · 2024
Cited alongside, same era.
Representation noising effectively prevents harmful fine-tuning on llms
D. Rosati, J. Wehner, K. Williams, L. Bartoszcze, D. Atanasov, R. Gonzales, S. Majumdar, C. Maple, H. Sajjad, and F. Rudzicz · 2024
Cited alongside, same era.
XSTest: A test suite for identifying exaggerated safety behaviours in large language models
P. Röttger, H. Kirk, B. Vidgen, G. Attanasio, F. Bianchi, and D. Hovy · 2024
Cited alongside, same era.
Great, now write an article about that: The crescendo multi-turn llm jailbreak attack
M. Russinovich, A. Salem, and R. Eldan · 2024
Cited alongside, same era.
Revisiting the robust alignment of circuit breakers, 2024
L. Schwinn and S. Geisler · 2024
Cited alongside, same era.
X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang · 2024
Cited alongside, same era.
A. Zou, L. Phan, J. Wang, D. Duenas, M. Lin, M. Andriushchenko, R. Wang, Z. Kolter, M. Fredrikson, and D. Hendrycks · 2024
Later among the works it cites.
Agentharm: A benchmark for measuring harmfulness of LLM agents
M. Andriushchenko, A. Souly, M. Dziemian, D. Duenas, M. Lin, J. Wang, D. Hendrycks, A. Zou, J. Z. Kolter, M. Fredrikson, Y. Gal, and X. Davies · 2025
Closest in time.
H. Cao, Y. Wang, S. Jing, Z. Peng, Z. Bai, Z. Cao, M. Fang, F. Feng, B. Wang, J. Liu, T. Yang, J. Huo, Y. Gao, F. Meng, X. Yang, C. Deng, and J. Feng · 2025
Closest in time.
OR-bench: An over-refusal benchmark for large language models, 2025
J. Cui, W.-L. Chiang, I. Stoica, and C.-J. Hsieh · 2025
Closest in time.
Deliberative alignment: Reasoning enables safer language models, 2025
M. Y. Guan, M. Joglekar, E. Wallace, S. Jain, B. Barak, A. Helyar, R. Dias, A. Vallone, H. Ren, J. Wei, H. W. Chung, S. Toyer, J. Heidecke, A. Beutel, and A. Glaese · 2025
Closest in time.
Virology capabilities test (vct): A multimodal virology q&a benchmark, 2025
J. Götting, P. Medeiros, J. G. Sanders, N. Li, L. Phan, K. Elabd, L. Justen, D. Hendrycks, and S. Donoughe · 2025
Closest in time.
Flex: A benchmark for evaluating robustness of fairness in large language models, 2025
D. Jung, S. Lee, H. Moon, C. Park, and H. Lim · 2025
Closest in time.
Jailbreaking to jailbreak, 2025
J. Kritz, V. Robinson, R. Vacareanu, B. Varjavand, M. Choi, B. Gogov, S. R. Team, S. Yue, W. E. Primack, and Z. Wang · 2025
Closest in time.
Elite: Enhanced language-image toxicity evaluation for safety, 2025
W. Lee, D. Lee, E. Choi, S. Yu, A. Yousefpour, H. Park, B. Ham, and S. Kim · 2025
Closest in time.
Longsafety: Evaluating long-context safety of large language models, 2025
Y. Lu, J. Cheng, Z. Zhang, S. Cui, C. Wang, X. Gu, Y. Dong, J. Tang, H. Wang, and M. Huang · 2025
Closest in time.
Safety pretraining: Toward the next generation of safe ai, 2025
P. Maini, S. Goyal, D. Sam, A. Robey, Y. Savani, Y. Jiang, A. Zou, Z. C. Lipton, and J. Z. Kolter · 2025
Closest in time.
V. M. G. Nair and V. V. Dantuluri · 2025
Closest in time.
Ai risk management framework, May 2025
NIST · 2025
Closest in time.
P. Rath, H. Shrawgi, P. Agrawal, and S. Dandapat · 2025
Closest in time.
M. Sharma, M. Tong, J. Mu, J. Wei, J. Kruthoff, S. Goodfriend, E. Ong, A. Peng, R. Agarwal, C. Anil, A. Askell, N. Bailey, J. Benton, E. Bluemke, S. R. Bowman, E. Christiansen, H. Cunningham, A. Dau, A. Gopal, R. Gilson, L. Graham, L. Howard, N. Kalra, T. Lee, K. Lin, P. Lofgren, F. Mosconi, C. O’Hara, C. Olsson, L. Petrini, S. Rajani, N. Saxena, A. Silverstein, T. Singh, T. Sumers, L. Tang, K. K. Troy, C. Weisser, R. Zhong, G. Zhou, J. Leike, J. Kaplan, and E. Perez · 2025
Closest in time.
Monitoring computer use via hierarchical summarization, 2025
T. Sumers, R. Agarwal, N. Bailey, T. Belonax, B. Clarke, J. Deng, E. Frondorf, K. Guru, K. Hankes, J. Klein, L. Lean, K. Lin, L. Petrini, M. Tucker, E. Perez, M. Sharma, and N. Saxena · 2025
Closest in time.
Sorry-bench: Systematically evaluating large language model safety refusal
T. Xie, X. Qi, Y. Zeng, Y. Huang, U. M. Sehwag, K. Huang, L. He, B. Wei, D. Li, Y. Sheng, R. Jia, B. Li, K. Li, D. Chen, P. Henderson, and P. Mittal · 2025
Closest in time.
AIR-BENCH 2024: A safety benchmark based on regulation and policies specified risk categories
Y. Zeng, Y. Yang, A. Zhou, J. Z. Tan, Y. Tu, Y. Mai, K. Klyman, M. Pan, R. Jia, D. Song, P. Liang, and B. Li · 2025
Closest in time.