Is Your LLM Outdated? A Deep Look at Temporal Generalization

Understand

The rapid advancement of Large Language Models (LLMs) has led to the development of benchmarks that consider temporal dynamics, however, there remains a gap in understanding how well these models can generalize across temporal contexts due to the inherent dynamic nature of language and information.

• This paper introduces the concept of temporal generalization in LLMs, including bias in past and future generalizations.
• Then we introduce FreshBench, a new evaluation framework that employs fresh text and event prediction for assessing LLMs' temporal adaptability, ensuring the evaluation process free from data leakage and subjective bias.
• The experiment shows significant temporal biases and a decline in performance over time.