Learning Distributions Generated by One-Layer ReLU Networks

Understand

We consider the problem of estimating the parameters of a $d$-dimensional rectified Gaussian distribution from i.i.d.

• samples.
• A rectified Gaussian distribution is defined by passing a standard Gaussian distribution through a one-layer ReLU neural network.
• We give a simple algorithm to estimate the parameters (i.e., the weight matrix and bias vector of the ReLU neural network) up to an error $\epsilon||W||_F$ using $\tilde{O}(1/\epsilon^2)$ samples and $\tilde{O}(d^2/\epsilon^2)$ time (log factors are ignored for simplicity).