2021

E(n) Equivariant Normalizing Flows

Satorras, Victor Garcia, Hoogeboom, Emiel, Fuchs, Fabian B. et al.

Understand

This paper introduces a generative model equivariant to Euclidean symmetries: E(n) Equivariant Normalizing Flows (E-NFs).

To construct E-NFs, we take the discriminative E(n) graph neural networks and integrate them as a differential equation to obtain an invertible equivariant function: a continuous-time normalizing flow.
We demonstrate that E-NFs considerably outperform baselines and existing methods from the literature on particle systems such as DW4 and LJ13, and on molecules from QM9 in terms of log-likelihood.
To the best of our knowledge, this is the first flow that jointly generates molecule features and positions in 3D.

Built on

Graph normalizing flows
Original
Liu, J., Kumar, A., Ba, J., Kiros, J., and Swersky, K. (2019) · 1905
Earlier work this paper cites.
Cormorant: Covariant molecular neural networks
Original
Anderson, B., Hy, T.-S., and Kondor, R. (2019) · 1906
Earlier work this paper cites.
Equivariant hamiltonian flows
Original
Rezende, D. J., Racanière, S., Higgins, I., and Toth, P. (2019) · 1909
Earlier work this paper cites.
Equivariant flows: sampling configurations for multi-body systems with symmetric energies
Original
Köhler, J., Klein, L., and Noé, F. (2019) · 1910
Earlier work this paper cites.
Neural canonical transformation with symplectic flows
Original
Li, S., Dong, C., Zhang, L., and Wang, L. (2019) · 1910
Earlier work this paper cites.
Divergence measures based on the shannon entropy
Lin, J. (1991) · 1991
Earlier work this paper cites.
Isometric transformation invariant and equivariant graph convolutional networks
Original
Horie, M., Morita, N., Ihara, Y., and Mitsume, N. (2020) · 2005
Earlier work this paper cites.
Se(3)-transformers: 3d roto-translation equivariant attention networks
Original
Fuchs, F. B., Worrall, D. E., Fischer, V., and Welling, M. (2020) · 2006
Earlier work this paper cites.
Survae flows: Surjections to bridge the gap between vaes and flows
Original
Nielsen, D., Jaini, P., Hoogeboom, E., Winther, O., and Welling, M. (2020) · 2007
Earlier work this paper cites.
Sampling using SU(N) gauge equivariant flows
Original
Boyda, D., Kanwar, G., Racanière, S., Rezende, D. J., Albergo, M. S., Cranmer, K., Hackett, D. C., and Shanahan, P. E. (2020) · 2008
Earlier work this paper cites.
Equivariant normalizing flows for point processes and sets
Original
Biloš, M. and Günnemann, S. (2020) · 2010
Earlier work this paper cites.
Lietransformer: Equivariant self-attention for lie groups
Original
Hutchinson, M., Lan, C. L., Zaidi, S., Dupont, E., Teh, Y. W., and Kim, H. (2020) · 2012
Earlier work this paper cites.
Spectral networks and locally connected networks on graphs
Original
Bruna, J., Zaremba, W., Szlam, A., and LeCun, Y. (2013) · 2013
Earlier work this paper cites.
High-dimensional probability estimation with deep density models
Original
Rippel, O. and Adams, R. P. (2013) · 2013
Earlier work this paper cites.

Similar

Quantum chemistry structures and properties of 134 kilo molecules
Ramakrishnan, R., Dral, P. O., Rupp, M., and Von Lilienfeld, O. A. (2014) · 2014
Cited alongside, same era.
NICE: Non-linear independent components estimation
Dinh, L., Krueger, D., and Bengio, Y. (2015) · 2015
Cited alongside, same era.
Variational Inference with Normalizing Flows
Rezende, D. and Mohamed, S. (2015) · 2015
Cited alongside, same era.
Group equivariant convolutional networks
Cohen, T. and Welling, M. (2016) · 2016
Cited alongside, same era.
Exploiting cyclic symmetry in convolutional neural networks
Dieleman, S., Fauw, J. D., and Kavukcuoglu, K. (2016) · 2016
Cited alongside, same era.
Automatic chemical design using a data-driven continuous representation of molecules
Original
Gómez-Bombarelli, R., Duvenaud, D., Hernández-Lobato, J. M., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., and Aspuru-Guzik, A. (2016) · 2016
Cited alongside, same era.
Semi-supervised classification with graph convolutional networks
Original
Kipf, T. N. and Welling, M. (2016) · 2016
Cited alongside, same era.
Continuous-time flows for deep generative models
Original
Chen, C., Li, C., Chen, L., Wang, W., Pu, Y., and Carin, L. (2017) · 2017
Cited alongside, same era.
Steerable cnns
Cohen, T. S. and Welling, M. (2017) · 2017
Cited alongside, same era.
Density estimation using Real NVP
Dinh, L., Sohl-Dickstein, J., and Bengio, S. (2017) · 2017
Cited alongside, same era.
Ffjord: Free-form continuous dynamics for scalable reversible generative models
Original
Grathwohl, W., Chen, R. T., Bettencourt, J., Sutskever, I., and Duvenaud, D. (2018) · 2018
Cited alongside, same era.
Glow: Generative flow with invertible 1x1 convolutions
Kingma, D. P. and Dhariwal, P. (2018) · 2018
Cited alongside, same era.

Then

Graphvae: Towards generation of small graphs using variational autoencoders
Simonovsky, M. and Komodakis, N. (2018) · 2018
Later among the works it cites.
Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds
Original
Thomas, N., Smidt, T., Kearnes, S., Yang, L., Li, L., Kohlhoff, K., and Riley, P. (2018) · 2018
Later among the works it cites.
Graphrnn: Generating realistic graphs with deep auto-regressive models
You, J., Ying, R., Ren, X., Hamilton, W. L., and Leskovec, J. (2018) · 2018
Later among the works it cites.
Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules
Gebauer, N. W. A., Gastegger, M., and Schütt, K. (2019) · 2019
Later among the works it cites.
Flow++: Improving flow-based generative models with variational dequantization and architecture design
Ho, J., Chen, X., Srinivas, A., Duan, Y., and Abbeel, P. (2019) · 2019
Later among the works it cites.
Efficient graph generation with graph recurrent attention networks
Liao, R., Li, Y., Song, Y., Wang, S., Hamilton, W. L., Duvenaud, D., Urtasun, R., and Zemel, R. S. (2019) · 2019
Later among the works it cites.
How to train your neural ODE: the world of jacobian and kinetic regularization
Finlay, C., Jacobsen, J., Nurbekyan, L., and Oberman, A. M. (2020) · 2020
Later among the works it cites.
Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data
Finzi, M., Stanton, S., Izmailov, P., and Wilson, A. G. (2020) · 2020
Later among the works it cites.
Directional message passing for molecular graphs
Klicpera, J., Groß, J., and Günnemann, S. (2020) · 2020
Later among the works it cites.
Equivariant flows: Exact likelihood generative learning for symmetric densities
Köhler, J., Klein, L., and Noé, F. (2020) · 2020
Later among the works it cites.
Group equivariant generative adversarial networks
Dey, N., Chen, A., and Ghafurian, S. (2021) · 2021
Closest in time.
Argmax flows and multinomial diffusion: Towards non-autoregressive language models
Original
Hoogeboom, E., Nielsen, D., Jaini, P., Forré, P., and Welling, M. (2021) · 2021
Closest in time.
E (n) equivariant graph neural networks
Original
Satorras, V. G., Hoogeboom, E., and Welling, M. (2021) · 2021
Closest in time.
Learning neural generative dynamics for molecular conformation generation
Original
Xu, M., Luo, S., Bengio, Y., Peng, J., and Tang, J. (2021) · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…