OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization

Understand

Structured pruning is a promising approach for reducing the inference costs of large vision and language models.

• By removing carefully chosen structures, e.g., neurons or attention heads, the improvements from this approach can be realized on standard deep learning hardware.
• In this work, we focus on structured pruning in the one-shot (post-training) setting, which does not require model retraining after pruning.
• We propose a novel combinatorial optimization framework for this problem, based on a layer-wise reconstruction objective and a careful reformulation that allows for scalable optimization.