Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue

Understand

Model editing is a technique that edits the large language models (LLMs) with updated knowledge to alleviate hallucinations without resource-intensive retraining.

• While current model editing methods can effectively modify a model's behavior within a specific area of interest, they often overlook the potential unintended side effects on the general abilities of LLMs such as reasoning, natural language inference, and question answering.
• In this paper, we raise concerns that model editing's improvements on factuality may come at the cost of a significant degradation of the model's general abilities.
• We systematically analyze the side effects by evaluating four popular editing methods on three LLMs across eight representative tasks.