How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients

Understand

As the post-training of large language models (LLMs) advances from instruction-following to complex reasoning tasks, understanding how different data affect finetuning dynamics remains largely unexplored.

• In this paper, we present a spectral analysis of layer-wise gradients induced by low/high-quality instruction and reasoning data for LLM post-training.
• Our analysis reveals that widely-studied metrics for data evaluation, e.g., IFD, InsTag, Difficulty, and Reward, can be explained and unified by spectral properties computed from gradients' singular value decomposition (SVD).
• Specifically, higher-quality data are usually associated with lower nuclear norms and higher effective ranks.