Statistical Linear Estimation with Penalized Estimators: an Application to Reinforcement Learning

Understand

Motivated by value function estimation in reinforcement learning, we study statistical linear inverse problems, i.e., problems where the coefficients of a linear system to be solved are observed in noise.

• We consider penalized estimators, where performance is evaluated using a matrix-weighted two-norm of the defect of the estimator measured with respect to the true, unknown coefficients.
• Two objective functions are considered depending whether the error of the defect measured with respect to the noisy coefficients is squared or unsquared.
• We propose simple, yet novel and theoretically well-founded data-dependent choices for the regularization parameters for both cases that avoid data-splitting.