2016

NIPS 2016 Tutorial: Generative Adversarial Networks

Goodfellow, Ian

Understand

This report summarizes the tutorial presented by the author at NIPS 2016 on generative adversarial networks (GANs).

The tutorial describes: (1) Why generative modeling is a topic worth studying, (2) how generative models work, and how GANs compare to other generative models, (3) the details of how GANs work, (4) research frontiers in GANs, and (5) state-of-the-art image models that combine GANs with other methods.
Finally, the tutorial contains three exercises for readers to complete, and the solutions to these exercises.

Built on

Massively parallel architectures for AI: NETL, thistle, and Boltzmann machines
Fahlman, S. E., Hinton, G. E., and Sejnowski, T. J. (1983) · 1983
Earlier work this paper cites.
Boltzmann machines: Constraint satisfaction networks that learn
Hinton, G. E., Sejnowski, T. J., and Ackley, D. H. (1984) · 1984
Earlier work this paper cites.
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., and Sejnowski, T. J. (1985) · 1985
Earlier work this paper cites.
Learning and relearning in Boltzmann machines
Hinton, G. E. and Sejnowski, T. J. (1986) · 1986
Earlier work this paper cites.
Simple statistical gradient-following algorithms connectionist reinforcement learning
Williams, R. J. (1992) · 1992
Earlier work this paper cites.
Higher order statistical decorrelation without information loss
Deco, G. and Brauer, W. (1995) · 1995
Earlier work this paper cites.
Does the wake-sleep algorithm learn good density estimators?
Frey, B. J., Hinton, G. E., and Dayan, P. (1996) · 1996
Earlier work this paper cites.
Graphical models for machine learning and digital communication
Frey, B. J. (1998) · 1998
Earlier work this paper cites.
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., and Teh, Y. (2006) · 2006
Earlier work this paper cites.
Learning multiple layers of representation
Hinton, G. E. (2007) · 2007
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009) · 2009
Earlier work this paper cites.
Deep Boltzmann machines
Salakhutdinov, R. and Hinton, G. (2009) · 2009
Earlier work this paper cites.
What does classifying more than 10,000 image categories tell us?
Deng, J., Berg, A. C., Li, K., and Fei-Fei, L. (2010) · 2010
Earlier work this paper cites.
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
Gutmann, M. and Hyvarinen, A. (2010) · 2010
Earlier work this paper cites.
Fast gradient-based inference with continuous latent variable models in auxiliary form
Original
Kingma, D. P. (2013) · 2013
Earlier work this paper cites.
Characterization and computation of local nash equilibria in continuous games
Ratliff, L. J., Burden, S. A., and Sastry, S. S. (2013) · 2013
Earlier work this paper cites.
Deep generative stochastic networks trainable by backprop
Bengio, Y., Thibodeau-Laufer, E., Alain, G., and Yosinski, J. (2014) · 2014
Earlier work this paper cites.
NICE: Non-linear independent components estimation
Original
Dinh, L., Krueger, D., and Bengio, Y. (2014) · 2014
Earlier work this paper cites.
On distinguishability criteria for estimating generative models
Goodfellow, I. J. (2014) · 2014
Earlier work this paper cites.
Generative adversarial networks
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014b) · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Kingma, D. and Ba, J. (2014) · 2014
Earlier work this paper cites.

Similar

Stochastic backpropagation and approximate inference in deep generative models
Original
Rezende, D. J., Mohamed, S., and Wierstra, D. (2014) · 2014
Cited alongside, same era.
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L. (2014) · 2014
Cited alongside, same era.
Intriguing properties of neural networks
Original
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. (2014) · 2014
Cited alongside, same era.
TensorFlow: Large-scale machine learning on heterogeneous systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X. (2015) · 2015
Cited alongside, same era.
Deep generative image models using a Laplacian pyramid of adversarial networks
Denton, E., Chintala, S., Szlam, A., and Fergus, R. (2015) · 2015
Cited alongside, same era.
Training generative neural networks via maximum mean discrepancy optimization
Original
Dziugaite, G. K., Roy, D. M., and Ghahramani, Z. (2015) · 2015
Cited alongside, same era.
Censoring representations with an adversary
Original
Edwards, H. and Storkey, A. (2015) · 2015
Cited alongside, same era.
Domain-adversarial training of neural networks
Original
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., and Lempitsky, V. (2015) · 2015
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Ioffe, S. and Szegedy, C. (2015) · 2015
Cited alongside, same era.
Generative moment matching networks
Original
Li, Y., Swersky, K., and Zemel, R. S. (2015) · 2015
Cited alongside, same era.
Unsupervised learning of visual structure using predictive generative networks
Original
Lotter, W., Kreiman, G., and Cox, D. (2015) · 2015
Cited alongside, same era.
Unsupervised representation learning with deep convolutional generative adversarial networks
Original
Radford, A., Metz, L., and Chintala, S. (2015) · 2015
Cited alongside, same era.

Then

Adversarially learned inference
Original
Dumoulin, V., Belghazi, I., Poole, B., Lamb, A., Arjovsky, M., Mastropietro, O., and Courville, A. (2016) · 2016
Closest in time.
Deep visual foresight for planning robot motion
Original
Finn, C. and Levine, S. (2016) · 2016
Closest in time.
Deep Learning
Goodfellow, I., Bengio, Y., and Courville, A. (2016) · 2016
Closest in time.
Generative adversarial imitation learning
Ho, J. and Ermon, S. (2016) · 2016
Closest in time.
Image-to-image translation with conditional adversarial networks
Original
Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A. A. (2016) · 2016
Closest in time.
Categorical reparameterization with gumbel-softmax
Original
Jang, E., Gu, S., and Poole, B. (2016) · 2016
Closest in time.
Improving variational inference with inverse autoregressive flow
Kingma, D. P., Salimans, T., and Welling, M. (2016) · 2016
Closest in time.
Photo-realistic single image super-resolution using a generative adversarial network
Original
Ledig, C., Theis, L., Huszar, F., Caballero, J., Aitken, A. P., Tejani, A., Totz, J., Wang, Z., and Shi, W. (2016) · 2016
Closest in time.
The concrete distribution: A continuous relaxation of discrete random variables
Original
Maddison, C. J., Mnih, A., and Teh, Y. W. (2016) · 2016
Closest in time.
Unrolled generative adversarial networks
Original
Metz, L., Poole, B., Pfau, D., and Sohl-Dickstein, J. (2016) · 2016
Closest in time.
Plug & play generative networks: Conditional iterative generation of images in latent space
Original
Nguyen, A., Yosinski, J., Bengio, Y., Dosovitskiy, A., and Clune, J. (2016) · 2016
Closest in time.
f-gan: Training generative neural samplers using variational divergence minimization
Original
Nowozin, S., Cseke, B., and Tomioka, R. (2016) · 2016
Closest in time.
Semi-supervised learning with generative adversarial networks
Original
Odena, A. (2016) · 2016
Closest in time.
Wavenet: A generative model for raw audio
Original
Oord, A. v. d., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016) · 2016
Closest in time.
Connecting generative adversarial networks and actor-critic methods
Original
Pfau, D. and Vinyals, O. (2016) · 2016
Closest in time.
Improved techniques for training gans
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. (2016) · 2016
Closest in time.
Mastering the game of go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al · 2016
Closest in time.
Adversarial perturbations of deep neural networks
Warde-Farley, D. and Goodfellow, I. (2016) · 2016
Closest in time.
On the quantitative analysis of decoder-based generative models
Original
Wu, Y., Burda, Y., Salakhutdinov, R., and Grosse, R. (2016) · 2016
Closest in time.
Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks
Original
Zhang, H., Xu, T., Li, H., Zhang, S., Huang, X., Wang, X., and Metaxas, D. (2016) · 2016
Closest in time.
Generative visual manipulation on the natural image manifold
Zhu, J.-Y., Krähenbühl, P., Shechtman, E., and Efros, A. A. (2016) · 2016
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…