Fetching the paper…
Reading the bibliography…
We study the effect of the stochastic gradient noise on the training of generative adversarial networks (GANs) and show that it can prevent the convergence of standard game optimization methods, while the batch version converges.
A stochastic approximation method
H. Robbins and S. Monro · 1951
Earlier work this paper cites.
The extragradient method for finding saddle points and other problems
G. Korpelevich · 1976
Earlier work this paper cites.
Finite-dimensional variational inequality and nonlinear complementarity problems: a survey of theory, algorithms and applications
P. T. Harker and J.-S. Pang · 1990
Earlier work this paper cites.
On linear convergence of iterative methods for the variational inequality problem
P. Tseng · 1995
Earlier work this paper cites.
Finite-Dimensional Variational Inequalities and Complementarity Problems Vol I
F. Facchinei and J.-S. Pang · 2003
Earlier work this paper cites.
Convex optimization
S. Boyd and L. Vandenberghe · 2004
Earlier work this paper cites.
Learning Multiple Layers of Features from Tiny Images
A. Krizhevsky · 2009
Earlier work this paper cites.
Large-scale machine learning with stochastic gradient descent
L. Bottou · 2010
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
X. Glorot and Y. Bengio · 2010
Earlier work this paper cites.
Solving variational inequalities with stochastic mirror-prox algorithm
A. Juditsky, A. Nemirovski, and C. Tauvel · 2011
Earlier work this paper cites.
Reading digits in natural images with unsupervised feature learning
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng · 2011
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
R. Johnson and T. Zhang · 2013
Earlier work this paper cites.
No more pesky learning rates
T. Schaul, S. Zhang, and Y. LeCun · 2013
Earlier work this paper cites.
Saga: A fast incremental gradient method with support for non-strongly convex composite objectives
A. Defazio, F. Bach, and S. Lacoste-Julien · 2014
Earlier work this paper cites.
Generative adversarial nets
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio · 2014
Cited alongside, same era.
A proximal stochastic gradient method with progressive variance reduction
L. Xiao and T. Zhang · 2014
Cited alongside, same era.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2015
Cited alongside, same era.
Variance reduced stochastic gradient descent with neighbors
T. Hofmann, A. Lucchi, S. Lacoste-Julien, and B. McWilliams · 2015
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
S. Ioffe and C. Szegedy · 2015
Cited alongside, same era.
Adam: A method for stochastic optimization
D. P. Kingma and J. Ba · 2015
GANs trained by a two time-scale update rule converge to a local nash equilibrium
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter · 2017
Later among the works it cites.
Extragradient method with variance reduction for stochastic variational inequalities
A. Iusem, A. Jofré, R. I. Oliveira, and P. Thompson · 2017
Later among the works it cites.
J. H. Lim and J. C. Ye · 2017
Later among the works it cites.
The numerics of GANs
L. Mescheder, S. Nowozin, and A. Geiger · 2017
Later among the works it cites.
Minimizing finite sums with the stochastic average gradient
M. Schmidt, N. Le Roux, and F. Bach · 2017
Later among the works it cites.
The marginal value of adaptive gradient methods in machine learning
A. C. Wilson, R. Roelofs, M. Stern, N. Srebro, and B. Recht · 2017
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
ImageNet Large Scale Visual Recognition Challenge
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei · 2015
Cited alongside, same era.
Rethinking the inception architecture for computer vision
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna · 2015
Cited alongside, same era.
Variance reduction for faster non-convex optimization
Z. Allen-Zhu and E. Hazan · 2016
Cited alongside, same era.
Smart: The stochastic monotone aggregated root-finding algorithm
D. Davis · 2016
Cited alongside, same era.
Stochastic variance reduction methods for saddle-point problems
B. Palaniappan and F. Bach · 2016
Cited alongside, same era.
Unsupervised representation learning with deep convolutional generative adversarial networks
A. Radford, L. Metz, and S. Chintala · 2016
Cited alongside, same era.
Later among the works it cites.
Training GANs with optimism
C. Daskalakis, A. Ilyas, V. Syrgkanis, and H. Zeng · 2018
Later among the works it cites.
On the ineffectiveness of variance reduced optimization for deep learning
A. Defazio and L. Bottou · 2018
Later among the works it cites.
Improved asynchronous parallel optimization analysis for stochastic incremental methods
R. Leblond, F. Pederegosa, and S. Lacoste-Julien · 2018
Later among the works it cites.
Spectral normalization for generative adversarial networks
T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida · 2018
Later among the works it cites.
Measuring the effects of data parallelism on neural network training
C. J. Shallue, J. Lee, J. Antognini, J. Sohl-Dickstein, R. Frostig, and G. E. Dahl · 2018
Later among the works it cites.
Self-Attention Generative Adversarial Networks
H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena · 2018
Later among the works it cites.
A tight and unified analysis of extragradient for a whole spectrum of differentiable games
W. Azizian, I. Mitliagkas, S. Lacoste-Julien, and G. Gidel · 2019
Closest in time.
Large scale GAN training for high fidelity natural image synthesis
A. Brock, J. Donahue, and K. Simonyan · 2019
Closest in time.