Dynamic Probabilistic Pruning: A general framework for hardware-constrained pruning at different granularities

Understand

Unstructured neural network pruning algorithms have achieved impressive compression rates.

• However, the resulting - typically irregular - sparse matrices hamper efficient hardware implementations, leading to additional memory usage and complex control logic that diminishes the benefits of unstructured pruning.
• This has spurred structured coarse-grained pruning solutions that prune entire filters or even layers, enabling efficient implementation at the expense of reduced flexibility.
• Here we propose a flexible new pruning mechanism that facilitates pruning at different granularities (weights, kernels, filters/feature maps), while retaining efficient memory organization (e.g.