Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference

Understand

Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference.

• In this work we perform an analytic study of the performances of one of them, the Langevin algorithm, in the context of noisy high-dimensional inference.
• We employ the Langevin algorithm to sample the posterior probability measure for the spiked matrix-tensor model.
• The typical behaviour of this algorithm is described by a system of integro-differential equations that we call the Langevin state evolution, whose solution is compared with the one of the state evolution of approximate message passing (AMP).