Fetching the paper…
Reading the bibliography…
We develop a general framework unifying several gradient-based stochastic optimization methods for empirical risk minimization problems both in centralized and distributed scenarios.
Introductory lectures on convex optimization: A basic course
Nesterov, Y. (2003) · 2003
Earlier work this paper cites.
Probability: A Graduate Course
Gut, A. (2005) · 2005
Earlier work this paper cites.
Distributed subgradient methods for multi-agent optimization
Nedic, A. and Ozdaglar, A. (2009) · 2009
Earlier work this paper cites.
Asynchronous gossip algorithms for stochastic optimization
Ram, S. S., Nedić, A., and Veeravalli, V. V. (2009) · 2009
Earlier work this paper cites.
Constrained consensus and optimization in multi-agent networks
Nedic, A., Ozdaglar, A., and Parrilo, P. A. (2010) · 2010
Earlier work this paper cites.
Discrete-time dynamic average consensus
Zhu, M. and Martínez, S. (2010) · 2010
Earlier work this paper cites.
Distributed optimization and statistical learning via the alternating direction method of multipliers
Boyd, S., Parikh, N., and Chu, E. (2011) · 2011
Earlier work this paper cites.
Distributed subgradient methods for convex optimization over random networks
Lobel, I. and Ozdaglar, A. (2011) · 2011
Earlier work this paper cites.
Large scale distributed deep networks
Dean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P., Yang, K., et al. (2012) · 2012
Earlier work this paper cites.
PMGT-VR: A decentralized proximal-gradient algorithmic framework with variance reduction
Ye, H., Xiong, W., and Zhang, T. (2020) · 2012
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
Johnson, R. and Zhang, T. (2013) · 2013
Earlier work this paper cites.
SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives
Defazio, A., Bach, F., and Lacoste-Julien, S. (2014) · 2014
Earlier work this paper cites.
Variance reduced stochastic gradient descent with neighbors
Hofmann, T., Lucchi, A., Lacoste-Julien, S., and McWilliams, B. (2015) · 2015
Earlier work this paper cites.
Asynchronous parallel stochastic gradient for nonconvex optimization
Lian, X., Huang, Y., Li, Y., and Liu, J. (2015) · 2015
Earlier work this paper cites.
Next: In-network nonconvex optimization
Di Lorenzo, P. and Scutari, G. (2016) · 2016
Earlier work this paper cites.
DSA: Decentralized double stochastic averaging gradient algorithm
Mokhtari, A. and Ribeiro, A. (2016) · 2016
Earlier work this paper cites.
On the convergence of decentralized gradient descent
Yuan, K., Ling, Q., and Yin, W. (2016) · 2016
Cited alongside, same era.
Distributed SAGA: Maintaining linear convergence rate with limited communication
Calauzenes, C. and Roux, N. L. (2017) · 2017
Cited alongside, same era.
Prox-PDA: The proximal primal-dual algorithm for fast distributed nonconvex optimization and learning over networks
Hong, M., Hajinezhad, D., and Zhao, M.-M. (2017) · 2017
Cited alongside, same era.
A unified analysis of stochastic optimization methods using jump system theory and quadratic constraints
Hu, B., Seiler, P., and Rantzer, A. (2017) · 2017
Cited alongside, same era.
Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent
Lian, X., Zhang, C., Zhang, H., Hsieh, C.-J., Zhang, W., and Liu, J. (2017) · 2017
Cited alongside, same era.
Tighter theory for local SGD on identical and heterogeneous data
Khaled, A., Mishchenko, K., and Richtárik, P. (2020) · 2020
Later among the works it cites.
A unified theory of decentralized SGD with changing topology and local updates
Koloskova, A., Loizou, N., Boreiri, S., Jaggi, M., and Stich, S. (2020) · 2020
Later among the works it cites.
Communication-efficient distributed optimization in networks with gradient tracking and variance reduction
Li, B., Cen, S., Chen, Y., and Chi, Y. (2020) · 2020
Later among the works it cites.
Distributed stochastic gradient tracking methods
Pu, S. and Nedić, A. (2020) · 2020
Later among the works it cites.
Push-pull gradient methods for distributed optimization in networks
Pu, S., Shi, W., Xu, J., and Nedic, A. (2020) · 2020
Later among the works it cites.
Improving the sample and communication complexity for decentralized non-convex optimization: Joint gradient estimation and tracking
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Achieving geometric convergence for distributed optimization over time-varying graphs
Nedic, A., Olshevsky, A., and Shi, W. (2017) · 2017
Cited alongside, same era.
SARAH: A novel method for machine learning problems using stochastic recursive gradient
Nguyen, L. M., Liu, J., Scheinberg, K., and Takáč, M. (2017) · 2017
Cited alongside, same era.
Harnessing smoothness to accelerate distributed optimization
Qu, G. and Li, N. (2017) · 2017
Cited alongside, same era.
Optimal algorithms for smooth and strongly convex distributed optimization in networks
Scaman, K., Bach, F., Bubeck, S., Lee, Y. T., and Massoulié, L. (2017) · 2017
Cited alongside, same era.
Network topology and communication-computation tradeoffs in decentralized optimization
Nedić, A., Olshevsky, A., and Rabbat, M. G. (2018) · 2018
Cited alongside, same era.
Stochastic gradient push for distributed deep learning
Assran, M., Loizou, N., Ballas, N., and Rabbat, M. (2019) · 2019
Cited alongside, same era.
On the convergence of FedAvg on non-iid data
Li, X., Huang, K., Yang, W., Wang, S., and Zhang, Z. (2019) · 2019
Cited alongside, same era.
Sun, H., Lu, S., and Hong, M. (2020) · 2020
Later among the works it cites.
Variance-reduced decentralized stochastic optimization with accelerated convergence
Xin, R., Khan, U. A., and Kar, S. (2020) · 2020
Later among the works it cites.
Accelerating gossip SGD with periodic global averaging
Chen, Y., Yuan, K., Zhang, Y., Pan, P., Xu, Y., and Yin, W. (2021) · 2021
Later among the works it cites.
Local SGD: Unified theory and new efficient methods
Gorbunov, E., Hanzely, F., and Richtárik, P. (2021) · 2021
Later among the works it cites.
An optimal algorithm for decentralized finite-sum optimization
Hendrikx, H., Bach, F., and Massoulie, L. (2021) · 2021
Later among the works it cites.
Li, B., Li, Z., and Chi, Y. (2021) · 2021
Later among the works it cites.
L-SVRG and L-Katyusha with arbitrary sampling
Qian, X., Qu, Z., and Richtárik, P. (2021) · 2021
Later among the works it cites.
Cooperative SGD: A unified framework for the design and analysis of local-update SGD algorithms
Wang, J. and Joshi, G. (2021) · 2021
Later among the works it cites.
Distributed stochastic gradient tracking algorithm with variance reduction for non-convex optimization
Jiang, X., Zeng, X., Sun, J., and Chen, J. (2022) · 2022
Closest in time.
Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes
Xu, J., Zhu, S., Soh, Y. C., and Xie, L. (2015) · 2060
Closest in time.