WikiFactDiff: A Large, Realistic, and Temporally Adaptable Dataset for Atomic Factual Knowledge Update in Causal Language Models

Understand

The factuality of large language model (LLMs) tends to decay over time since events posterior to their training are "unknown" to them.

• One way to keep models up-to-date could be factual update: the task of inserting, replacing, or removing certain simple (atomic) facts within the model.
• To study this task, we present WikiFactDiff, a dataset that describes the evolution of factual knowledge between two dates as a collection of simple facts divided into three categories: new, obsolete, and static.
• We describe several update scenarios arising from various combinations of these three types of basic update.