Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models

Understand

Conversational large language models are trained to refuse to answer harmful questions.

• However, emergent jailbreaking techniques can still elicit unsafe outputs, presenting an ongoing challenge for model alignment.
• To better understand how different jailbreak types circumvent safeguards, this paper analyses model activations on different jailbreak inputs.
• We find that it is possible to extract a jailbreak vector from a single class of jailbreaks that works to mitigate jailbreak effectiveness from other semantically-dissimilar classes.