Denoising Noisy Neural Networks: A Bayesian Approach with Compensation

Understand

Deep neural networks (DNNs) with noisy weights, which we refer to as noisy neural networks (NoisyNNs), arise from the training and inference of DNNs in the presence of noise.

• NoisyNNs emerge in many new applications, including the wireless transmission of DNNs, the efficient deployment or storage of DNNs in analog devices, and the truncation or quantization of DNN weights.
• This paper studies a fundamental problem of NoisyNNs: how to reconstruct the DNN weights from their noisy manifestations.
• While all prior works relied on the maximum likelihood (ML) estimation, this paper puts forth a denoising approach to reconstruct DNNs with the aim of maximizing the inference accuracy of the reconstructed models.