Bypassing the Safety Training of Open-Source LLMs with Priming Attacks

Understand

With the recent surge in popularity of LLMs has come an ever-increasing need for LLM safety training.

• In this paper, we investigate the fragility of SOTA open-source LLMs under simple, optimization-free attacks we refer to as $\textit{priming attacks}$, which are easy to execute and effectively bypass alignment from safety training.
• Our proposed attack improves the Attack Success Rate on Harmful Behaviors, as measured by Llama Guard, by up to $3.3\times$ compared to baselines.
• Source code and data are available at https://github.com/uiuc-focal-lab/llm-priming-attacks.