Proximal Adam: Robust Adaptive Update Scheme for Constrained Optimization

Phuong TT, Phong LT (2019) On the convergence proof of AMSGrad and a new version DOI 10.1109/ACCESS.2019.2916341

Original

1904

Earlier work this paper cites.

Moreau JJ (1965) Proximité et dualité dans un espace hilbertien. Bulletin de la Société Mathématique de France 93:273–299, DOI 10.24033/bsmf.1625

1965

Earlier work this paper cites.

Lupton R, Blanton M, Fekete G, Hogg D, O’Mullane W, Szalay A, Wherry N (2004) Preparing Red-Green-Blue images from CCD data. Publications of the Astronomical Society of the Pacific 116(816):133–137, DOI 10.1086/382245

2004

Earlier work this paper cites.

Hunter JD (2007) Matplotlib: A 2d graphics environment. Computing in Science Engineering 9(3):90–95

2007

Earlier work this paper cites.

Beck A, Teboulle M (2009) A fast iterative Shrinkage-Thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences 2(1):183–202, DOI 10.1137/080716542

2009

Earlier work this paper cites.

Combettes PL, Dũng Đ, Vũ BC (2011) Proximity for sums of composite functions. Journal of mathematical analysis and applications 380(2):680–688, DOI 10.1016/j.jmaa.2011.02.079

2011

Earlier work this paper cites.

Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research: JMLR 12(Jul):2121–2159, URL http://www.jmlr.org/papers/v12/duchi11a.html

2011

Earlier work this paper cites.

Schmidt M, Roux NL, Bach FR (2011) Convergence rates of inexact Proximal-Gradient methods for convex optimization. In: Shawe-Taylor J, Zemel RS, Bartlett PL, Pereira F, Weinberger KQ (eds) Advances in Neural Information Processing Systems 24, Curran Associates, Inc., pp 1458–1466, URL http://papers.nips.cc/paper/4452-convergence-rates-of-inexact-proximal-gradient-methods-for-convex-optimization.pdf

2011

Earlier work this paper cites.

van der Walt S, Colbert SC, Varoquaux G (2011) The numpy array: A structure for efficient numerical computation. Computing in Science Engineering 13(2):22–30

2011

Cited alongside, same era.

Becker S, Fadili J (2012) A quasi-newton proximal splitting method. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in Neural Information Processing Systems 25, Curran Associates, Inc., pp 2618–2626, URL http://papers.nips.cc/paper/4523-a-quasi-newton-proximal-splitting-method.pdf

2012

Cited alongside, same era.

Hinton G, Srivastava N, Swersky K (2012) Neural networks for machine learning. Tech. rep., URL https://www.cs.toronto.edu/˜tijmen/csc321/slides/lecture_slides_lec6.pdf

2012

Cited alongside, same era.

Rapin J, Bobin J, Larue A, Starck J (2013) Sparse and Non-Negative BSS for noisy data. IEEE transactions on signal processing: a publication of the IEEE Signal Processing Society 61(22):5620–5632, DOI 10.1109/TSP.2013.2279358

2013

Cited alongside, same era.

Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, URL http://arxiv.org/abs/1412.6980

Original

2015

Later among the works it cites.

Tran-Dinh Q, Kyrillidis A, Cevher V (2015) Composite self-concordant minimization. Journal of machine learning research: JMLR URL http://www.jmlr.org/papers/volume16/trandihn15a/trandihn15a.pdf

2015

Later among the works it cites.

Chen J, Gu Q (2018) Closing the generalization gap of adaptive gradient methods in training deep neural networks URL http://arxiv.org/abs/1806.06763 , 1806.06763

Original

2018

Later among the works it cites.

Melchior P, Moolekamp F, Jerdee M, Armstrong R, Sun AL, Bosch J, Lupton R (2018) scarlet: Source separation in multi-band images by constrained matrix factorization. Astronomy and Computing 24:129–142, DOI 10.1016/j.ascom.2018.07.001

2018

Later among the works it cites.