Calibrating the Rigged Lottery: Making All Tickets Reliable

Understand

Although sparse training has been successfully used in various resource-limited deep learning tasks to save memory, accelerate training, and reduce inference time, the reliability of the produced sparse models remains unexplored.

• Previous research has shown that deep neural networks tend to be over-confident, and we find that sparse training exacerbates this problem.
• Therefore, calibrating the sparse models is crucial for reliable prediction and decision-making.
• In this paper, we propose a new sparse training method to produce sparse models with improved confidence calibration.