Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection

Understand

Detecting Out-of-distribution (OOD) inputs have been a critical issue for neural networks in the open world.

• However, the unstable behavior of OOD detection along the optimization trajectory during training has not been explored clearly.
• In this paper, we first find the performance of OOD detection suffers from overfitting and instability during training: 1) the performance could decrease when the training error is near zero, and 2) the performance would vary sharply in the final stage of training.
• Based on our findings, we propose Average of Pruning (AoP), consisting of model averaging and pruning, to mitigate the unstable behaviors.