Double Descent and Overfitting under Noisy Inputs and Distribution Shift for Linear Denoisers

Understand

Despite the importance of denoising in modern machine learning and ample empirical work on supervised denoising, its theoretical understanding is still relatively scarce.

• One concern about studying supervised denoising is that one might not always have noiseless training data from the test distribution.
• It is more reasonable to have access to noiseless training data from a different dataset than the test dataset.
• Motivated by this, we study supervised denoising and noisy-input regression under distribution shift.