Fluctuation-based Adaptive Structured Pruning for Large Language Models

Understand

Network Pruning is a promising way to address the huge computing resource demands of the deployment and inference of Large Language Models (LLMs).

• Retraining-free is important for LLMs' pruning methods.
• However, almost all of the existing retraining-free pruning approaches for LLMs focus on unstructured pruning, which requires specific hardware support for acceleration.
• In this paper, we propose a novel retraining-free structured pruning framework for LLMs, named FLAP (FLuctuation-based Adaptive Structured Pruning).