On the Role of Attention Heads in Large Language Model Safety

Understand

Large language models (LLMs) achieve state-of-the-art performance on multiple language tasks, yet their safety guardrails can be circumvented, leading to harmful generations.

• In light of this, recent research on safety mechanisms has emerged, revealing that when safety representations or component are suppressed, the safety capability of LLMs are compromised.
• However, existing research tends to overlook the safety impact of multi-head attention mechanisms, despite their crucial role in various model functionalities.
• Hence, in this paper, we aim to explore the connection between standard attention mechanisms and safety capability to fill this gap in the safety-related mechanistic interpretability.