Greedy Output Approximation: Towards Efficient Structured Pruning for LLMs Without Retraining

Understand

To remove redundant components of large language models (LLMs) without incurring significant computational costs, this work focuses on single-shot pruning without a retraining phase.

• We simplify the pruning process for Transformer-based LLMs by identifying a depth-2 pruning structure that functions independently.
• Additionally, we propose two inference-aware pruning criteria derived from the optimization perspective of output approximation, which outperforms traditional training-aware metrics such as gradient and Hessian.
• We also introduce a two-step reconstruction technique to mitigate pruning errors without model retraining.