LookAhead Tuning: Safer Language Models via Partial Answer Previews

Understand

Fine-tuning enables large language models (LLMs) to adapt to specific domains, but often compromises their previously established safety alignment.

• To mitigate the degradation of model safety during fine-tuning, we introduce LookAhead Tuning, a lightweight and effective data-driven approach that preserves safety during fine-tuning.
• The method introduces two simple strategies that modify training data by previewing partial answer prefixes, thereby minimizing perturbations to the model's initial token distributions and maintaining its built-in safety mechanisms.
• Comprehensive experiments demonstrate that LookAhead Tuning effectively maintains model safety without sacrificing robust performance on downstream tasks.