2020

Latent Variable Modelling with Hyperbolic Normalizing Flows

Bose, Avishek Joey, Smofsky, Ariella, Liao, Renjie et al.

Understand

The choice of approximate posterior distributions plays a central role in stochastic variational inference (SVI).

One effective solution is the use of normalizing flows \cut{defined on Euclidean spaces} to construct flexible posterior distributions.
However, one key limitation of existing normalizing flows is that they are restricted to the Euclidean space and are ill-equipped to model data with an underlying hierarchical structure.
To address this fundamental limitation, we present the first extension of normalizing flows to hyperbolic spaces.

Built on

Infectious diseases of humans: dynamics and control
Anderson, R. M., Anderson, B., and May, R. M · 1992
Earlier work this paper cites.
Foundations of Hyperbolic Manifolds
Ratcliffe, J. G · 1994
Earlier work this paper cites.
Polyominoes: puzzles, patterns, problems, and packings , volume 16
Golomb, S. W · 1996
Earlier work this paper cites.
Intrinsic statistics on riemannian manifolds: Basic tools for geometric measurements
Pennec, X · 2006
Earlier work this paper cites.
The human disease network
Goh, K.-I., Cusick, M. E., Valle, D., Childs, B., Vidal, M., and Barabási, A.-L · 2007
Earlier work this paper cites.
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G · 2008
Earlier work this paper cites.
Low distortion delaunay embedding of trees in hyperbolic plane
Sarkar, R · 2011
Earlier work this paper cites.
Stochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J · 2013
Earlier work this paper cites.
Auto-encoding variational bayes
Original
Kingma, D. P. and Welling, M · 2013
Earlier work this paper cites.
Generative adversarial nets
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Kingma, D. P. and Ba, J · 2014
Earlier work this paper cites.
New riemannian priors on the univariate normal model
Said, S., Bombrun, L., and Berthoumieu, Y · 2014
Earlier work this paper cites.
Importance weighted autoencoders
Original
Burda, Y., Grosse, R., and Salakhutdinov, R · 2015
Earlier work this paper cites.
Adversarial autoencoders
Original
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., and Frey, B · 2015
Earlier work this paper cites.
Variational inference with normalizing flows
Rezende, D. J. and Mohamed, S · 2015
Earlier work this paper cites.
Empirical evaluation of rectified activations in convolutional network
Original
Xu, B., Wang, N., Chen, T., and Li, M · 2015
Earlier work this paper cites.
Normalizing flows on riemannian manifolds
Original
Gemici, M. C., Rezende, D., and Mohamed, S · 2016
Earlier work this paper cites.

Similar

Variational graph auto-encoders
Original
Kipf, T. N. and Welling, M · 2016
Cited alongside, same era.
Revisiting semi-supervised learning with graph embeddings
Original
Yang, Z., Cohen, W. W., and Salakhutdinov, R · 2016
Cited alongside, same era.
Neural embeddings of graphs in hyperbolic space
Original
Chamberlain, B. P., Clough, J., and Deisenroth, M. P · 2017
Cited alongside, same era.
Density estimation using real nvp
Dinh, L., Sohl-Dickstein, J., and Bengio, S · 2017
Cited alongside, same era.
Gradient estimators for implicit models
Original
Li, Y. and Turner, R. E · 2017
Cited alongside, same era.
Poincaré embeddings for learning hierarchical representations
Nickel, M. and Kiela, D · 2017
Cited alongside, same era.
Kernel implicit variational inference
Original
Shi, J., Sun, S., and Zhu, J · 2017
Cited alongside, same era.
Wasserstein auto-encoders
Original
Tolstikhin, I., Bousquet, O., Gelly, S., and Schoelkopf, B · 2017
Cited alongside, same era.
Graph attention networks
Original
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y · 2017
Cited alongside, same era.
Hyperspherical variational auto-encoders
Original
Davidson, T. R., Falorsi, L., De Cao, N., Kipf, T., and Tomczak, J. M · 2018
Cited alongside, same era.
Embedding text in hyperbolic spaces
Original
Dhingra, B., Shallue, C. J., Norouzi, M., Dai, A. M., and Dahl, G. E · 2018
Cited alongside, same era.
Hyperbolic neural networks
Ganea, O., Bécigneul, G., and Hofmann, T · 2018
Cited alongside, same era.

Then

Hyperbolic representation learning for fast and efficient neural question answering
Tay, Y., Tuan, L. A., and Hui, S. C · 2018
Later among the works it cites.
Hyperbolic graph convolutional neural networks
Chami, I., Ying, Z., Ré, C., and Leskovec, J · 2019
Later among the works it cites.
Block neural autoregressive flow
Original
De Cao, N., Titov, I., and Aziz, W · 2019
Later among the works it cites.
Reparameterizing distributions on lie groups
Original
Falorsi, L., de Haan, P., Davidson, T. R., and Forré, P · 2019
Later among the works it cites.
Adversarial autoencoders with constant-curvature latent manifolds
Grattarola, D., Livi, L., and Alippi, C · 2019
Later among the works it cites.
Hyperbolic image embeddings
Original
Khrulkov, V., Mirvakhabova, L., Ustinova, E., Oseledets, I., and Lempitsky, V · 2019
Later among the works it cites.
Normalizing flows: Introduction and ideas
Original
Kobyzev, I., Prince, S., and Brubaker, M. A · 2019
Later among the works it cites.
Lorentzian distance learning for hyperbolic representations
Law, M., Liao, R., Snell, J., and Zemel, R · 2019
Later among the works it cites.
Efficient graph generation with graph recurrent attention networks
Liao, R., Li, Y., Song, Y., Wang, S., Hamilton, W., Duvenaud, D. K., Urtasun, R., and Zemel, R · 2019
Later among the works it cites.
Continuous hierarchical representations with poincaré variational auto-encoders
Mathieu, E., Le Lan, C., Maddison, C. J., Tomioka, R., and Teh, Y. W · 2019
Later among the works it cites.
A wrapped normal distribution on hyperbolic space for gradient-based learning
Nagano, Y., Yamaguchi, S., Fujita, Y., and Koyama, M · 2019
Later among the works it cites.
Poincar \ \backslash ’e wasserstein autoencoder
Original
Ovinnikov, I · 2019
Later among the works it cites.
Normalizing flows for probabilistic modeling and inference
Original
Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., and Lakshminarayanan, B · 2019
Later among the works it cites.
Mixed-curvature variational autoencoders
Original
Skopek, O., Ganea, O.-E., and Bécigneul, G · 2019
Later among the works it cites.
Numerically accurate hyperbolic embeddings using tiling-based models
Yu, T. and De Sa, C. M · 2019
Later among the works it cites.
Flows for simultaneous manifold learning and density estimation
Original
Brehmer, J. and Cranmer, K · 2020
Closest in time.
Normalizing flows on tori and spheres
Original
Rezende, D. J., Papamakarios, G., Racanière, S., Albergo, M. S., Kanwar, G., Shanahan, P. E., and Cranmer, K · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…