Fetching the paper…

Reading the bibliography…

2018

Mining gold from implicit models to improve likelihood-free inference

Brehmer, Johann, Louppe, Gilles, Pavez, Juan et al.

Original
PDF
Post on X

Understand

Simulators often provide the best description of real-world phenomena.

However, they also lead to challenging inverse problems because the density they implicitly define is often intractable.
We present a new suite of simulation-based inference techniques that go beyond the traditional Approximate Bayesian Computation approach, which struggles in a high-dimensional setting, and extend methods that use surrogate models based on neural networks.
We show that additional information, such as the joint likelihood ratio and the joint score, can often be extracted from simulators and used to augment the training data for these surrogate models.

Mining gold from implicit models to improve likelihood-free inference · Around

Built on

J. Hermans, V. Begy, and G. Louppe: ‘Likelihood-free MCMC with Approximate Likelihood Ratios’ , 2019. arXiv:1903.04057
Original
1903
Earlier work this paper cites.
J. Alsing and B. Wandelt: ‘Nuisance hardened data compression for fast likelihood-free inference’ , 2019. arXiv:1903.01473
Original
1903
Earlier work this paper cites.
D. S. Greenberg, M. Nonnenmacher, and J. H. Macke: ‘Automatic Posterior Transformation for Likelihood-Free Inference’. arXiv e-prints arXiv:1905.07488, 2019. arXiv:1905.07488
Original
1905
Earlier work this paper cites.
A. J. Lotka: ‘Analytical note on certain rhythmic relations in organic systems’. Proceedings of the National Academy of Sciences 6 (7), p. 410, 1920
1920
Earlier work this paper cites.
A. J. Lotka: ‘Undamped oscillations derived from the law of mass action.’ Journal of the american chemical society 42 (8), p. 1595, 1920
1920
Earlier work this paper cites.
D. T. Gillespie: ‘A general method for numerically simulating the stochastic time evolution of coupled chemical reactions’. Journal of Computational Physics 22 (4), p. 403 , 1976
1976
Earlier work this paper cites.
D. B. Rubin: ‘Bayesianly justifiable and relevant frequency calculations for the applied statistician’. Ann. Statist. 12 (4), p. 1151, 1984. URL https://doi.org/10.1214/aos/1176346785
1984
Earlier work this paper cites.
R. J. Williams: ‘Simple statistical gradient-following algorithms for connectionist reinforcement learning’. In ‘Reinforcement Learning’, Springer, p. 5–32, 1992
1992
Earlier work this paper cites.
M. A. Beaumont, W. Zhang, and D. J. Balding: ‘Approximate bayesian computation in population genetics’. Genetics 162 (4), p. 2025, 2002
2002
Earlier work this paper cites.
K. Cranmer and T. Plehn: ‘Maximum significance at the LHC and Higgs decays to muons’. Eur. Phys. J. C51, p. 415, 2007. arXiv:hep-ph/0605268
2007
Earlier work this paper cites.
Y. Fan, D. J. Nott, and S. A. Sisson: ‘Approximate Bayesian Computation via Regression Density Estimation’. ArXiv e-prints , 2012. arXiv:1212.1479
Original
2012
Earlier work this paper cites.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, et al.: ‘Generative Adversarial Networks’. ArXiv e-prints , 2014. arXiv:1406.2661
Original
2014
Earlier work this paper cites.
L. Dinh, D. Krueger, and Y. Bengio: ‘NICE: Non-linear Independent Components Estimation’. ArXiv e-prints , 2014. arXiv:1410.8516
Original
2014
Earlier work this paper cites.
F. Wood, J. W. van de Meent, and V. Mansinghka: ‘A new approach to probabilistic programming inference’. In ‘Proceedings of the 17th International conference on Artificial Intelligence and Statistics’, p. 1024-1032, 2014
2014
Earlier work this paper cites.
J. Alwall, R. Frederix, S. Frixione, et al.: ‘The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations’. JHEP 07, p. 079, 2014. arXiv:1405.0301
Original
2014
Earlier work this paper cites.
T. Plehn, P. Schichtel, and D. Wiegand: ‘Where boosted significances come from’. Phys. Rev. D89 (5), p. 054002, 2014. arXiv:1311.2591
Original
2014
Earlier work this paper cites.
K. Cranmer, J. Pavez, and G. Louppe: ‘Approximating Likelihood Ratios with Calibrated Discriminative Classifiers’ , 2015. arXiv:1506.02169
Original
2015
Earlier work this paper cites.
D. Jimenez Rezende and S. Mohamed: ‘Variational Inference with Normalizing Flows’. ArXiv e-prints , 2015. arXiv:1505.05770
Original
2015
Earlier work this paper cites.
M. Germain, K. Gregor, I. Murray, and H. Larochelle: ‘MADE: Masked Autoencoder for Distribution Estimation’. ArXiv e-prints , 2015. arXiv:1502.03509
Original
2015
Earlier work this paper cites.
A. Gelman, D. Lee, and J. Guo: ‘Stan: A Probabilistic Programming Language for Bayesian Inference and Optimization’. Journal of Educational and Behavioral Statistics 40 (5), p. 530, 2015
2015
Earlier work this paper cites.

Similar

K. Cranmer and G. Louppe: ‘Unifying generative models and exact likelihood-free inference with conditional bijections’. J. Brief Ideas , 2016
2016
Cited alongside, same era.
G. Louppe, K. Cranmer, and J. Pavez: ‘carl: a likelihood-free inference toolbox’. J. Open Source Softw. , 2016
2016
Cited alongside, same era.
S. Mohamed and B. Lakshminarayanan: ‘Learning in Implicit Generative Models’. ArXiv e-prints , 2016. arXiv:1610.03483
Original
2016
Cited alongside, same era.
L. Dinh, J. Sohl-Dickstein, and S. Bengio: ‘Density estimation using Real NVP’. ArXiv e-prints , 2016. arXiv:1605.08803
Original
2016
Cited alongside, same era.
B. Uria, M.-A. Côté, K. Gregor, I. Murray, and H. Larochelle: ‘Neural Autoregressive Distribution Estimation’. ArXiv e-prints , 2016. arXiv:1605.02226
Original

Then

F. Kling, T. Plehn, and P. Schichtel: ‘Maximizing the significance in Higgs boson pair analyses’. Phys. Rev. D95 (3), p. 035026, 2017. arXiv:1607.07441
Original
2017
Later among the works it cites.
B. Eli, J. P. Chen, M. Jankowiak, et al.: ‘Pyro: Deep probabilistic programming’. https://github.com/uber/pyro , 2017
2017
Later among the works it cites.
D. Tran, M. D. Hoffman, R. A. Saurous, E. Brevdo, K. Murphy, and D. M. Blei: ‘Deep probabilistic programming’. arXiv preprint arXiv:1701.03757 , 2017
Original
2017
Later among the works it cites.
N. Siddharth, B. Paige, J.-W. van de Meent, et al.: ‘Learning disentangled representations with semi-supervised deep generative models’. In I. Guyon, U. V. Luxburg, S. Bengio, et al. (eds.), ‘Advances in Neural Information Processing Systems 30’, p. 5927–5937. Curran Associates, Inc., 2017
2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2016

Cited alongside, same era.

A. van den Oord, S. Dieleman, H. Zen, et al.: ‘WaveNet: A Generative Model for Raw Audio’. ArXiv e-prints , 2016. arXiv:1609.03499

2016

Cited alongside, same era.

A. van den Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt, A. Graves, and K. Kavukcuoglu: ‘Conditional Image Generation with PixelCNN Decoders’. ArXiv e-prints , 2016. arXiv:1606.05328

2016

Cited alongside, same era.

A. van den Oord, N. Kalchbrenner, and K. Kavukcuoglu: ‘Pixel Recurrent Neural Networks’. ArXiv e-prints , 2016. arXiv:1601.06759

2016

Cited alongside, same era.

G. Papamakarios and I. Murray: ‘Fast ε \varepsilon -free inference of simulation models with bayesian conditional density estimation’. In ‘Advances in Neural Information Processing Systems’, p. 1028–1036, 2016

2016

Cited alongside, same era.

B. Paige and F. Wood: ‘Inference Networks for Sequential Monte Carlo in Graphical Models’. ArXiv e-prints , 2016. arXiv:1602.06701

2016

Cited alongside, same era.

R. Dutta, J. Corander, S. Kaski, and M. U. Gutmann: ‘Likelihood-free inference by ratio estimation’. ArXiv e-prints , 2016. arXiv:1611.10242

2016

Cited alongside, same era.

P. Baldi, K. Cranmer, T. Faucett, P. Sadowski, and D. Whiteson: ‘Parameterized neural networks for high-energy physics’. Eur. Phys. J. C76 (5), p. 235, 2016. arXiv:1601.07913

2016

Cited alongside, same era.

J. Alsing, B. Wandelt, and S. Feeney: ‘Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology’ , 2018. arXiv:1801.01497

2018

Closest in time.

T. Charnock, G. Lavaux, and B. D. Wandelt: ‘Automatic physical inference with information maximizing neural networks’. Phys. Rev. D97 (8), p. 083004, 2018. arXiv:1802.03537

2018

Closest in time.

T. Dinev and M. U. Gutmann: ‘Dynamic Likelihood-free Inference via Ratio Estimation (DIRE)’. arXiv e-prints arXiv:1810.09899, 2018. arXiv:1810.09899

2018

Closest in time.

C.-W. Huang, D. Krueger, A. Lacoste, and A. Courville: ‘Neural Autoregressive Flows’. ArXiv e-prints , 2018. arXiv:1804.00779

2018

Closest in time.

G. Papamakarios, D. C. Sterratt, and I. Murray: ‘Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows’. ArXiv e-prints , 2018. arXiv:1805.07226

2018

Closest in time.

T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud: ‘Neural ordinary differential equations’. CoRR abs/1806.07366, 2018. arXiv:1806.07366 , URL http://arxiv.org/abs/1806.07366

2018

Closest in time.

D. P. Kingma and P. Dhariwal: ‘Glow: Generative Flow with Invertible 1x1 Convolutions’. arXiv e-prints arXiv:1807.03039, 2018. arXiv:1807.03039

2018

Closest in time.

W. Grathwohl, R. T. Q. Chen, J. Bettencourt, I. Sutskever, and D. Duvenaud: ‘FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models’. ArXiv e-prints , 2018. arXiv:1810.01367

2018

Closest in time.

J.-M. Lueckmann, G. Bassetto, T. Karaletsos, and J. H. Macke: ‘Likelihood-free inference with emulator networks’. arXiv e-prints arXiv:1805.09294, 2018. arXiv:1805.09294

2018

Closest in time.

J. Brehmer, K. Cranmer, G. Louppe, and J. Pavez: ‘Constraining Effective Field Theories with Machine Learning’. Phys. Rev. Lett. 121 (11), p. 111801, 2018. arXiv:1805.00013

2018

Closest in time.

J. Brehmer, K. Cranmer, G. Louppe, and J. Pavez: ‘A Guide to Constraining Effective Field Theories with Machine Learning’. Phys. Rev. D98 (5), p. 052004, 2018. arXiv:1805.00020

2018

Closest in time.

J. Brehmer, K. Cranmer, G. Louppe, and J. Pavez: ‘Code repository for the generalized Galton board example in the paper “Mining gold from implicit models to improve likelihood-free inference”’. http://github.com/johannbrehmer/simulator-mining-example , 2018

2018

Closest in time.

E. Bingham, J. P. Chen, M. Jankowiak, et al.: ‘Pyro: Deep Universal Probabilistic Programming’. Journal of Machine Learning Research , 2018

2018

Closest in time.

J. Alsing and B. Wandelt: ‘Generalized massive optimal data compression’. Mon. Not. Roy. Astron. Soc. 476 (1), p. L60, 2018. arXiv:1712.00012

2018

Closest in time.

J. Brehmer, K. Cranmer, G. Louppe, and J. Pavez: ‘Code repository for the Lotka-Volterra example in the paper “Mining gold from implicit models to improve likelihood-free inference”’. http://github.com/johannbrehmer/goldmine , 2018

2018

Closest in time.

Participants of the Likelihood-Free Inference Meeting at the Flatiron Institute 2019: ‘Code repository for the automatic calculation of joint score and joint likelihood ratio with Pyro.’ https://github.com/LFITaskForce/benchmark , 2019

2019

Closest in time.