Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data

Understand

Large scale deep learning provides a tremendous opportunity to improve the quality of content recommendation systems by employing both wider and deeper models, but this comes at great infrastructural cost and carbon footprint in modern data centers.

• Pruning is an effective technique that reduces both memory and compute demand for model inference.
• However, pruning for online recommendation systems is challenging due to the continuous data distribution shift (a.k.a non-stationary data).
• Although incremental training on the full model is able to adapt to the non-stationary data, directly applying it on the pruned model leads to accuracy loss.