DeepTwist: Learning Model Compression via Occasional Weight Distortion

Understand

Model compression has been introduced to reduce the required hardware resources while maintaining the model accuracy.

• Lots of techniques for model compression, such as pruning, quantization, and low-rank approximation, have been suggested along with different inference implementation characteristics.
• Adopting model compression is, however, still challenging because the design complexity of model compression is rapidly increasing due to additional hyper-parameters and computation overhead in order to achieve a high compression ratio.
• In this paper, we propose a simple and efficient model compression framework called DeepTwist which distorts weights in an occasional manner without modifying the underlying training algorithms.