2020

Normalizing Flows on Tori and Spheres

Rezende, Danilo Jimenez, Papamakarios, George, Racanière, Sébastien et al.

Understand

Normalizing flows are a powerful tool for building expressive distributions in high dimensions.

So far, most of the literature has concentrated on learning flows on Euclidean spaces.
Some problems however, such as those involving angles, are defined on spaces with more complex geometries, such as tori or spheres.
In this paper, we propose and compare expressive and numerically stable flows on such spaces.

Built on

Dispersion on a sphere
Fisher, R · 1953
Earlier work this paper cites.
Foundations of differential geometry , volume 1
Kobayashi, S. and Nomizu, K · 1963
Earlier work this paper cites.
Orientation statistics
Downs, T. D · 1972
Earlier work this paper cites.
The Fisher–Bingham distribution on the sphere
Kent, J. T · 1982
Earlier work this paper cites.
Sequential Monte Carlo methods in practice
Arnaud, D., de Freitas, N., and Gordon, N · 2001
Earlier work this paper cites.
Fitting mixtures of Kent distributions to aid in joint set identification
Peel, D., Whiten, W. J., and McLachlan, G. J · 2001
Earlier work this paper cites.
Circular regression
Downs, T. D. and Mardia, K. V · 2002
Earlier work this paper cites.
Generalized Euler angle parametrization for 𝑆𝑈 ⁡ ( N ) \mathit{SU}(N)
Tilma, T. and Sudarshan, E · 2002
Earlier work this paper cites.
Generalized Euler angle parameterization for U ⁡ ( N ) \mathit{U}(N) with applications to 𝑆𝑈 ⁡ ( N ) \mathit{SU}(N) coset volume measures
Tilma, T. and Sudarshan, E · 2004
Earlier work this paper cites.
Sampling realistic protein conformations using local structural bias
Hamelryck, T., Kent, J. T., and Krogh, A · 2006
Earlier work this paper cites.
The generalized von Mises distribution
Gatto, R. and Jammalamadaka, S. R · 2007
Earlier work this paper cites.
Protein bioinformatics and mixtures of bivariate von Mises distributions for angular data
Mardia, K. V., Taylor, C. C., and Subramaniam, G. K · 2007
Earlier work this paper cites.
A generative, probabilistic model of local protein structure
Boomsma, W., Mardia, K. V., Taylor, C. C., Ferkinghoff-Borg, J., Krogh, A., and Hamelryck, T · 2008
Earlier work this paper cites.
A circular–circular regression model
Kato, S., Shimizu, K., and Shieh, G. S · 2008
Earlier work this paper cites.
Monte Carlo strategies in scientific computing
Liu, J. S · 2008
Earlier work this paper cites.

Similar

A multivariate von Mises distribution with applications to bioinformatics
Mardia, K. V., Hughes, G., Taylor, C. C., and Singh, H · 2008
Cited alongside, same era.
Directional statistics
Mardia, K. V. and Jupp, P. E · 2009
Cited alongside, same era.
A smoothed backbone-dependent rotamer library for proteins derived from adaptive kernel density estimates and regressions
Shapovalov, M. V. and Dunbrack Jr, R. L · 2011
Cited alongside, same era.
Numerically stable sampling of the von Mises Fisher distribution on S 2 S^{2} (and other tricks)
Jakob, W · 2012
Cited alongside, same era.
Spatial analysis of wave direction data using wrapped Gaussian processes
Jona-Lasinio, G., Gelfand, A., and Jona-Lasinio, M · 2012
Cited alongside, same era.
Rigid motion estimation using mixtures of projected Gaussians
Feiten, W., Lang, M., and Hirche, S · 2013
Cited alongside, same era.
A Jacobian inequality for gradient maps on the sphere and its application to directional statistics
Sei, T · 2013
Cited alongside, same era.
A family of nonparametric density estimation algorithms
Tabak, E. G. and Turner, C. V · 2013
Cited alongside, same era.
Directional data analysis under the general projected normal distribution
Wang, F. and Gelfand, A. E · 2013
Cited alongside, same era.
Extensions of probability distributions on torus, cylinder and disc
Wang, M · 2013
Cited alongside, same era.
Möbius transformation and a Cauchy family on the sphere
Original
Kato, S. and McCullagh, P · 2015
Cited alongside, same era.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2015
Cited alongside, same era.

Then

Improved variational inference with inverse autoregressive flow
Kingma, D. P., Salimans, T., Jozefowicz, R., Chen, X., Sutskever, I., and Welling, M · 2016
Later among the works it cites.
Density estimation using Real NVP
Dinh, L., Sohl-Dickstein, J., and Bengio, S · 2017
Later among the works it cites.
The multivariate generalised von Mises distribution: Inference and applications
Navarro, A. K. W., Frellsen, J., and Turner, R. E · 2017
Later among the works it cites.
Masked autoregressive flow for density estimation
Papamakarios, G., Pavlakou, T., and Murray, I · 2017
Later among the works it cites.
Quantum theory, groups and representations: An introduction
Woit, P · 2017
Later among the works it cites.
Hyperspherical variational auto-encoders
Davidson, T. R., Falorsi, L., De Cao, N., Kipf, T., and Tomczak, J. M · 2018
Later among the works it cites.
Explorations in homeomorphic variational auto-encoding
Falorsi, L., de Haan, P., Davidson, T. R., De Cao, N., Weiler, M., Forré, P., and Cohen, T. S · 2018
Later among the works it cites.
Directional grid maps: modeling multimodal angular uncertainty in dynamic environments
Senanayake, R. and Ramos, F · 2018
Later among the works it cites.
Reparameterizing distributions on Lie groups
Falorsi, L., de Haan, P., Davidson, T. R., and Forré, P · 2019
Later among the works it cites.
Continuous hierarchical representations with Poincaré variational auto-encoders
Mathieu, E., Le Lan, C., Maddison, C. J., Tomioka, R., and Teh, Y. W · 2019
Later among the works it cites.
Neural importance sampling
Müller, T., McWilliams, B., Rousselle, F., Gross, M., and Novák, J · 2019
Later among the works it cites.
Normalizing flows for probabilistic modeling and inference
Original
Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., and Lakshminarayanan, B · 2019
Later among the works it cites.
Riemannian normalizing flow on variational Wasserstein autoencoder for text modeling
Wang, P. Z. and Wang, W. Y · 2019
Later among the works it cites.
A Wasserstein minimum velocity approach to learning unnormalized models
Wang, Z., Cheng, S., Li, Y., Zhu, J., and Zhang, B · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…