Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator

Understand

We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges' estimator.

• The oracle property is often a consequence of sparsity of an estimator.
• We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity whenever the loss function is unbounded.
• For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting.