Fetching the paper…

Reading the bibliography…

2020

Decentralized Langevin Dynamics

Kungurtsev, Vyacheslav

Original
PDF
Post on X

Understand

Langevin MCMC gradient optimization is a class of increasingly popular methods for estimating a posterior distribution.

This paper addresses the algorithm as applied in a decentralized setting, wherein data is distributed across a network of agents which act to cooperatively solve the problem using peer-to-peer gossip communication.
We show, theoretically, results in 1) the time-complexity to $\epsilon$-consensus for the continuous time stochastic differential equation, 2) convergence rate in $L^2$ norm to consensus for the discrete implementation as defined by the Euler-Maruyama discretization and 3) convergence rate in the Wasserstein metric to the optimal stationary distribution for the discretized dynamics.

Decentralized Langevin Dynamics · Around

Built on

Introduction to optimization
Boris T Polyak · 1987
Earlier work this paper cites.
Bayesian learning via stochastic gradient langevin dynamics
Max Welling and Yee W Teh · 2011
Earlier work this paper cites.
Distributed stochastic gradient mcmc
Sungjin Ahn, Babak Shahbaba, and Max Welling · 2014
Earlier work this paper cites.
A complete recipe for stochastic gradient mcmc
Yi-An Ma, Tianqi Chen, and Emily Fox · 2015
Earlier work this paper cites.

Similar

Bridging the gap between stochastic gradient mcmc and stochastic optimization
Changyou Chen, David Carlson, Zhe Gan, Chunyuan Li, and Lawrence Carin · 2016
Cited alongside, same era.
Next: In-network nonconvex optimization
Paolo Di Lorenzo and Gesualdo Scutari · 2016
Cited alongside, same era.
On nonconvex decentralized gradient descent
Jinshan Zeng and Wotao Yin · 2018
Cited alongside, same era.
User-friendly guarantees for the langevin monte carlo with inaccurate gradient
Arnak S Dalalyan and Avetik Karagulyan · 2019
Cited alongside, same era.

Then

Annealing for distributed global optimization
Original
Brian Swenson, Soummya Kar, H Vincent Poor, and Jose’MF Moura · 2019
Later among the works it cites.
Distributed gradient descent: Nonconvergence to saddle points and the stable-manifold theorem
Original
Brian Swenson, Ryan Murray, H Vincent Poor, and Soummya Kar · 2019
Later among the works it cites.
Distributed learning in non-convex environments–part i: Agreement at a linear rate
Original
Stefan Vlaski and Ali H Sayed · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…