Stein's unbiased risk estimate and Hyv\"arinen's score matching

Understand

Given a collection of observed signals corrupted with Gaussian noise, how can we learn to optimally denoise them? This fundamental problem arises in both empirical Bayes and generative modeling.

• In empirical Bayes, the predominant approach is via nonparametric maximum likelihood estimation (NPMLE), while in generative modeling, score matching (SM) methods have proven very successful.
• In our setting, Hyv\"arinen's implicit SM is equivalent to another classical idea from statistics -- Stein's Unbiased Risk Estimate (SURE).
• Revisiting SURE minimization, we establish, for the first time, that SURE achieves nearly parametric rates of convergence of the regret in the classical empirical Bayes setting with homoscedastic noise.