An analytic comparison of regularization methods for Gaussian Processes

Understand

Gaussian Processes (GPs) are a popular approach to predict the output of a parameterized experiment.

• They have many applications in the field of Computer Experiments, in particular to perform sensitivity analysis, adaptive design of experiments and global optimization.
• Nearly all of the applications of GPs require the inversion of a covariance matrix that, in practice, is often ill-conditioned.
• Regularization methodologies are then employed with consequences on the GPs that need to be better understood.The two principal methods to deal with ill-conditioned covariance matrices are i) pseudoinverse and ii) adding a positive constant to the diagonal (the so-called nugget regularization).The first part of this paper provides an algebraic comparison of PI and nugget regularizations.