Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes

Understand

Structured pruning is a promising approach to create smaller, faster large language models.

• However, existing methods typically rely on computing the gradient via backward passes, which can inflate memory requirements and compute costs.
• In this work we introduce Bonsai, a gradient-free structured pruning method that eliminates the need for backpropagation, significantly reducing memory requirements and compute costs while achieving state-of-the-art pruning performance.
• Bonsai uses forward-pass-only perturbative pruning to enable efficient compression of large models on a broader range of hardware configurations.