AttentionDefense: Leveraging System Prompt Attention for Explainable Defense Against Novel Jailbreaks

Understand

In the past few years, Language Models (LMs) have shown par-human capabilities in several domains.

• Despite their practical applications and exceeding user consumption, they are susceptible to jailbreaks when malicious input exploits the LM's weaknesses, causing it to deviate from its intended behavior.
• Current defensive strategies either classify the input prompt as adversarial or prevent LMs from generating harmful outputs.
• However, it is challenging to explain the reason behind the malicious nature of the jailbreak, which results in a wide variety of closed-box approaches.