Token-wise Influential Training Data Retrieval for Large Language Models

Understand

Given a Large Language Model (LLM) generation, how can we identify which training data led to this generation? In this paper, we proposed RapidIn, a scalable framework adapting to LLMs for estimating the influence of each training data.

• The proposed framework consists of two stages: caching and retrieval.
• First, we compress the gradient vectors by over 200,000x, allowing them to be cached on disk or in GPU/CPU memory.
• Then, given a generation, RapidIn efficiently traverses the cached gradients to estimate the influence within minutes, achieving over a 6,326x speedup.