2017

A-NICE-MC: Adversarial Training for MCMC

Song, Jiaming, Zhao, Shengjia, Ermon, Stefano

Understand

Existing Markov Chain Monte Carlo (MCMC) methods are either based on general-purpose and domain-agnostic schemes which can lead to slow convergence, or hand-crafting of problem-specific proposals by an expert.

We propose A-NICE-MC, a novel method to train flexible parametric Markov chain kernels to produce samples with desired properties.
First, we propose an efficient likelihood-free adversarial training method to train a Markov chain and mimic a given data distribution.
Then, we leverage flexible volume preserving flows to obtain parametric kernels for MCMC.

Built on

W. K. Hastings, “Monte carlo sampling methods using markov chains and their applications,” Biometrika
1970
Earlier work this paper cites.
R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Machine learning
1992
Earlier work this paper cites.
CRC press, 1994
B. Efron and R. J. Tibshirani, An introduction to the bootstrap · 1994
Earlier work this paper cites.
P. J. Green, “Reversible jump markov chain monte carlo computation and bayesian model determination,” Biometrika
1995
Earlier work this paper cites.
S. P. Brooks and A. Gelman, “General methods for monitoring convergence of iterative simulations,” Journal of computational and graphical statistics
1998
Earlier work this paper cites.
N. De Freitas, P. Højen-Sørensen, M. I. Jordan, and S. Russell, “Variational mcmc,” in Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence
2001
Earlier work this paper cites.
S. Boyd, P. Diaconis, and L. Xiao, “Fastest mixing markov chain on a graph,” SIAM review
2004
Earlier work this paper cites.
R. M. Neal et al
2011
Earlier work this paper cites.
M. Girolami and B. Calderhead, “Riemann manifold langevin and hamiltonian monte carlo methods,” Journal of the Royal Statistical Society: Series B (Statistical Methodology)
2011
Earlier work this paper cites.
N. Mahendran, Z. Wang, F. Hamze, and N. De Freitas, “Adaptive mcmc with bayesian optimization.,” in AISTATS
2012
Earlier work this paper cites.
W. Jakob and S. Marschner, “Manifold exploration: a markov chain monte carlo technique for rendering scenes with difficult specular transport,” ACM Transactions on Graphics (TOG)
2012
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114
Original
2013
Earlier work this paper cites.
R. Ranganath, S. Gerrish, and D. Blei, “Black box variational inference,” in Artificial Intelligence and Statistics
2014
Earlier work this paper cites.

Similar

D. J. Rezende, S. Mohamed, and D. Wierstra, “Stochastic backpropagation and approximate inference in deep generative models,” arXiv preprint arXiv:1401.4082
Original
2014
Cited alongside, same era.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems
2014
Cited alongside, same era.
S. Ermon, C. P. Gomes, A. Sabharwal, and B. Selman, “Designing fast absorbing markov chains.,” in AAAI
2014
Cited alongside, same era.
L. Dinh, D. Krueger, and Y. Bengio, “Nice: Non-linear independent components estimation,” arXiv preprint arXiv:1410.8516
Original
2014
Cited alongside, same era.
Y. Bengio, E. Thibodeau-Laufer, G. Alain, and J. Yosinski, “Deep generative stochastic networks trainable by backprop,” 2014
2014
Cited alongside, same era.
Cambridge university press, 2014
D. P. Landau and K. Binder, A guide to Monte Carlo simulations in statistical physics · 2014
Cited alongside, same era.
M. D. Hoffman and A. Gelman, “The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo.,” Journal of Machine Learning Research
2014
Cited alongside, same era.
D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980
Original
2014
Cited alongside, same era.
T. Salimans, D. P. Kingma, M. Welling, et al
2015
Cited alongside, same era.
J. Gorham and L. Mackey, “Measuring sample quality with stein’s method,” in Advances in Neural Information Processing Systems
2015
Cited alongside, same era.
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434
Original
2015
Cited alongside, same era.
R. K. Srivastava, K. Greff, and J. Schmidhuber, “Highway networks,” arXiv preprint arXiv:1505.00387
Original
2015
Cited alongside, same era.

Then

Y. Li, K. Swersky, and R. Zemel, “Generative moment matching networks,” in International Conference on Machine Learning
2015
Later among the works it cites.
S. Mohamed and B. Lakshminarayanan, “Learning in implicit generative models,” arXiv preprint arXiv:1610.03483
Original
2016
Later among the works it cites.
J. Gorham, A. B. Duncan, S. J. Vollmer, and L. Mackey, “Measuring sample quality with diffusions,” arXiv preprint arXiv:1611.06972
Original
2016
Later among the works it cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
2016
Later among the works it cites.
D. P. Kingma, T. Salimans, and M. Welling, “Improving variational inference with inverse autoregressive flow,” arXiv preprint arXiv:1606.04934
Original
2016
Later among the works it cites.
T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” in Advances in Neural Information Processing Systems
2016
Later among the works it cites.
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al
Original
2016
Later among the works it cites.
J. Gorham and L. Mackey, “Measuring sample quality with kernels,” arXiv preprint arXiv:1703.01717
Original
2017
Closest in time.
M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” arXiv preprint arXiv:1701.07875
Original
2017
Closest in time.
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, “Improved training of wasserstein gans,” arXiv preprint arXiv:1704.00028
Original
2017
Closest in time.
F. Bordes, S. Honari, and P. Vincent, “Learning to generate samples from noise through infusion training,” ICLR
2017
Closest in time.
A. Grover, M. Dhar, and S. Ermon, “Flow-gan: Bridging implicit and prescribed learning in generative models,” arXiv preprint arXiv:1705.08868
Original
2017
Closest in time.
J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” arXiv preprint arXiv:1703.10593
Original
2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…