Fetching the paper…
Reading the bibliography…
We consider the decentralized stochastic optimization problems, where a network of $n$ nodes, each owning a local cost function, cooperate to find a minimizer of the globally-averaged cost.
J. Tsitsiklis, D. Bertsekas, and M. Athans, “Distributed asynchronous deterministic and stochastic gradient optimization algorithms,” IEEE transactions on automatic control
1986
Earlier work this paper cites.
C. G. Lopes and A. H. Sayed, “Diffusion least-mean squares over adaptive networks: Formulation and performance analysis,” IEEE Transactions on Signal Processing
2008
Earlier work this paper cites.
A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Transactions on Automatic Control
2009
Earlier work this paper cites.
M. Zinkevich, M. Weimer, L. Li, and A. J. Smola, “Parallelized stochastic gradient descent,” in Advances in neural information processing systems
2010
Earlier work this paper cites.
A. Smola and S. Narayanamurthy, “An architecture for parallel topic models,” Proceedings of the VLDB Endowment
2010
Earlier work this paper cites.
J. C. Duchi, A. Agarwal, and M. J. Wainwright, “Dual averaging for distributed optimization: Convergence analysis and network scaling,” IEEE Transactions on Automatic control
2011
Earlier work this paper cites.
J. Liu and A. S. Morse, “Accelerated linear iterations for distributed averaging,” Annual Reviews in Control
2011
Earlier work this paper cites.
J. Chen and A. H. Sayed, “Diffusion adaptation strategies for distributed optimization and learning over networks,” IEEE Transactions on Signal Processing
2012
Earlier work this paper cites.
E. Wei and A. Ozdaglar, “Distributed alternating direction method of multipliers,” in IEEE Conference on Decision and Control (CDC)
2012
Earlier work this paper cites.
L. Deng, “The mnist database of handwritten digit images for machine learning research [best of the web],” IEEE Signal Processing Magazine
2012
Earlier work this paper cites.
J. Chen and A. H. Sayed, “Distributed pareto optimization via diffusion strategies,” IEEE Journal of Selected Topics in Signal Processing
2013
Earlier work this paper cites.
A. H. Sayed, “Adaptation, learning, and optimization over networks,” Foundations and Trends in Machine Learning
2014
Earlier work this paper cites.
W. Shi, Q. Ling, K. Yuan, G. Wu, and W. Yin, “On the linear convergence of the admm in decentralized consensus optimization,” IEEE Transactions on Signal Processing
2014
Earlier work this paper cites.
J. Xu, S. Zhu, Y. C. Soh, and L. Xie, “Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes,” in IEEE Conference on Decision and Control (CDC)
2015
Earlier work this paper cites.
W. Shi, Q. Ling, G. Wu, and W. Yin, “EXTRA: An exact first-order algorithm for decentralized consensus optimization,” SIAM Journal on Optimization
2015
Earlier work this paper cites.
R. Rossi and N. Ahmed, “The network data repository with interactive graph analytics and visualization,” in Twenty-Ninth AAAI Conference on Artificial Intelligence
2015
Earlier work this paper cites.
P. Di Lorenzo and G. Scutari, “Next: In-network nonconvex optimization,” IEEE Transactions on Signal and Information Processing over Networks
2016
Earlier work this paper cites.
K. Yuan, Q. Ling, and W. Yin, “On the convergence of decentralized gradient descent,” SIAM Journal on Optimization
2016
Earlier work this paper cites.
X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent,” in Advances in Neural Information Processing Systems
2017
Earlier work this paper cites.
A. Nedic, A. Olshevsky, and W. Shi, “Achieving geometric convergence for distributed optimization over time-varying graphs,” SIAM Journal on Optimization
2017
Cited alongside, same era.
K. Scaman, F. Bach, S. Bubeck, Y. T. Lee, and L. Massoulié, “Optimal algorithms for smooth and strongly convex distributed optimization in networks,” in International Conference on Machine Learning
2017
Cited alongside, same era.
X. Lian, W. Zhang, C. Zhang, and J. Liu, “Asynchronous decentralized parallel stochastic gradient descent,” in International Conference on Machine Learning
2018
Cited alongside, same era.
H. Tang, X. Lian, M. Yan, C. Zhang, and J. Liu, “ d 2 d^{2} : Decentralized training over decentralized data,” in International Conference on Machine Learning
2018
Cited alongside, same era.
K. Yuan, B. Ying, X. Zhao, and A. H. Sayed, “Exact dffusion for distributed optimization and learning – Part I: Algorithm development,” IEEE Transactions on Signal Processing
K. Yuan, S. A. Alghunaim, B. Ying, and A. H. Sayed, “On the influence of bias-correction on distributed stochastic optimization,” IEEE Transactions on Signal Processing
2020
Later among the works it cites.
S. Pu and A. Nedić, “Distributed stochastic gradient tracking methods,” Mathematical Programming
2020
Later among the works it cites.
C. A. Uribe, S. Lee, A. Gasnikov, and A. Nedić, “A dual approach for optimal algorithms in distributed optimization over networks,” Optimization Methods and Software
2020
Later among the works it cites.
H. Li, C. Fang, W. Yin, and Z. Lin, “Decentralized accelerated gradient methods with increasing penalty parameters,” IEEE Transactions on Signal Processing
2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2018
Cited alongside, same era.
K. Yuan, B. Ying, X. Zhao, and A. H. Sayed, “Exact diffusion for distributed optimization and learning—Part II: Convergence analysis,” IEEE Transactions on Signal Processing
2018
Cited alongside, same era.
G. Qu and N. Li, “Harnessing smoothness to accelerate distributed optimization,” IEEE Transactions on Control of Network Systems
2018
Cited alongside, same era.
K. Scaman, F. Bach, S. Bubeck, L. Massoulié, and Y. T. Lee, “Optimal algorithms for non-smooth distributed optimization in networks,” in Advances in Neural Information Processing Systems
2018
Cited alongside, same era.
A. S. Berahas, R. Bollapragada, N. S. Keskar, and E. Wei, “Balancing communication and computation in distributed optimization,” IEEE Transactions on Automatic Control
2018
Cited alongside, same era.
M. Assran, N. Loizou, N. Ballas, and M. Rabbat, “Stochastic gradient push for distributed deep learning,” in International Conference on Machine Learning (ICML)
2019
Cited alongside, same era.
Z. Li, W. Shi, and M. Yan, “A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates,” IEEE Transactions on Signal Processing
2019
Cited alongside, same era.
S. Lu, X. Zhang, H. Sun, and M. Hong, “Gnsd: A gradient-tracking based nonconvex stochastic algorithm for decentralized optimization,” in 2019 IEEE Data Science Workshop (DSW)
2019
Cited alongside, same era.
2021
Closest in time.
K. Yuan, Y. Chen, X. Huang, Y. Zhang, P. Pan, Y. Xu, and W. Yin, “DecentLaM: Decentralized momentum sgd for large-batch deep training,” pp. 3029–3039, 2021
2021
Closest in time.
B. Ying, K. Yuan, Y. Chen, H. Hu, P. Pan, and W. Yin, “Exponential graph is provably efficient for decentralized deep training,” in Advances in Neural Information Processing Systems (NeurIPS)
2021
Closest in time.
Accessed: 2021-05-15
B. Ying, K. Yuan, H. Hu, Y. Chen, and W. Yin, “BlueFog: Make decentralized algorithms practical for optimization and deep learning.” https://github.com/Bluefog-Lib/bluefog , 2021 · 2021
Closest in time.
Y. Chen, K. Yuan, Y. Zhang, P. Pan, Y. Xu, and W. Yin, “Accelerating gossip sgd with periodic global averaging,” in International Conference on Machine Learning (ICML)
2021
Closest in time.
S. Pu, A. Olshevsky, and I. C. Paschalidis, “A sharp estimate on the transient time of distributed stochastic gradient descent,” IEEE Transactions On Automatic Control, early access
2021
Closest in time.
2021
Closest in time.
R. Xin, U. A. Khan, and S. Kar, “An improved convergence analysis for decentralized online stochastic non-convex optimization,” IEEE Transactions on Signal Processing
2021
Closest in time.
2021
Closest in time.
A. Koloskova, T. Lin, and S. U. Stich, “An improved analysis of gradient tracking for decentralized machine learning,” Advances in Neural Information Processing Systems
2021
Closest in time.
S. A. Alghunaim, E. K. Ryu, K. Yuan, and A. H. Sayed, “Decentralized proximal gradient algorithms with linear convergence rates,” IEEE Transactions on Automatic Control
2021
Closest in time.
2021
Closest in time.
Y. Lu and C. De Sa, “Optimal complexity in decentralized training,” in International Conference on Machine Learning
2021
Closest in time.
R. Xin, U. A. Khan, and S. Kar, “Fast decentralized nonconvex finite-sum optimization with recursive variance reduction,” SIAM Journal on Optimization
2022
Closest in time.