Fetching the paper…
Reading the bibliography…
Given large datasets and sufficient compute, is it beneficial to design neural architectures for the structure and symmetries of each problem? Or is it more efficient to learn them from data? We study empirically how equivariant and non-equivariant networks scale with compute and training samples.
Feature spaces which admit and detect invariant signal transformations
Shun-ichi Amari · 1978
Earlier work this paper cites.
Scaling and generalization in neural networks: a case study
Subutai Ahmad and Gerald Tesauro · 1988
Earlier work this paper cites.
On the limited memory bfgs method for large scale optimization
Dong C Liu and Jorge Nocedal · 1989
Earlier work this paper cites.
Robust estimation of a location parameter
Peter J Huber · 1992
Earlier work this paper cites.
Representation theory and invariant neural networks
Jeffrey Wood and John Shawe-Taylor · 1996
Earlier work this paper cites.
Correspondence-free structure from motion
Ameesh Makadia, Christopher Geyer, and Kostas Daniilidis · 2007
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma · 2015
Earlier work this paper cites.
Group equivariant convolutional networks
Taco Cohen and Max Welling · 2016
Earlier work this paper cites.
A probabilistic data-driven model for planar pushing
Maria Bauza and Alberto Rodriguez · 2017
Earlier work this paper cites.
Deep learning scaling is predictable, empirically
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou · 2017
Earlier work this paper cites.
Generalization error of invariant classifiers
Jure Sokolic, Raja Giryes, Guillermo Sapiro, and Miguel Rodrigues · 2017
Earlier work this paper cites.
Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Roto-translation covariant convolutional networks for medical image analysis
Erik J Bekkers, Maxime W Lafarge, Mitko Veta, Koen AJ Eppenhof, Josien PW Pluim, and Remco Duits · 2018
Earlier work this paper cites.
Rotation equivariant CNNs for digital pathology
Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling · 2018
Earlier work this paper cites.
3d G-CNNs for pulmonary nodule detection
Marysia Winkels and Taco S Cohen · 2018
Earlier work this paper cites.
Improved semantic segmentation for histopathology using rotation equivariant convolutional networks
Jim Winkens, Jasper Linmans, Bastiaan S Veeling, Taco S Cohen, and Max Welling · 2018
Earlier work this paper cites.
Adaptive input representations for neural language modeling
Alexei Baevski and Michael Auli · 2019
Earlier work this paper cites.
Fast transformer decoding: One write-head is all you need
Noam Shazeer · 2019
Earlier work this paper cites.
A guided tour to the plane-based geometric algebra pga, 2020
Leo Dorst · 2020
Earlier work this paper cites.
Scaling laws for autoregressive generative modeling
Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al · 2020
Earlier work this paper cites.
Deep-neural-network solution of the electronic schrödinger equation
Jan Hermann, Zeno Schätzle, and Frank Noé · 2020
Earlier work this paper cites.
Scaling laws for neural language models
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei · 2020
Earlier work this paper cites.
On the benefits of invariance in neural networks
Clare Lyle, Mark van der Wilk, Marta Kwiatkowska, Yarin Gal, and Benjamin Bloem-Reddy · 2020
Earlier work this paper cites.
A data and compute efficient design for limited-resources deep learning
Mirgahney Mohamed, Gabriele Cesa, Taco S Cohen, and Max Welling · 2020
Earlier work this paper cites.
Ab initio solution of the many-electron schrödinger equation with deep neural networks
David Pfau, James S Spencer, Alexander GDG Matthews, and W Matthew C Foulkes · 2020
Cited alongside, same era.
A constructive prediction of the generalization error across scales
Jonathan S Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, and Nir Shavit · 2020
Cited alongside, same era.
Fourier features let networks learn high frequency functions in low dimensional domains
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng · 2020
Cited alongside, same era.
Sampling using su (n) gauge equivariant flows
Denis Boyda, Gurtej Kanwar, Sébastien Racanière, Danilo Jimenez Rezende, Michael S Albergo, Kyle Cranmer, Daniel C Hackett, and Phiala E Shanahan · 2021
Cited alongside, same era.
Geometric deep learning: Grids, groups, graphs, geodesics, and gauges
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković · 2021
Cited alongside, same era.
Equiformer: Equivariant graph attention transformer for 3d atomistic graphs
Yi-Lun Liao and Tess Smidt · 2023
Later among the works it cites.
Scaling data-constrained language models
Niklas Muennighoff, Alexander M Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, and Colin Raffel · 2023
Later among the works it cites.
Learning local equivariant representations for large-scale atomistic dynamics
Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J Owen, Mordechai Kornbluth, and Boris Kozinsky · 2023
Later among the works it cites.
Scaling laws vs model architectures: How does inductive bias influence scaling?
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran, Dani Yogatama, and Donald Metzler · 2023
Later among the works it cites.
Generating molecular conformer fields
Yuyang Wang, Ahmed AA Elhag, Navdeep Jaitly, Joshua M Susskind, and Miguel Ángel Bautista · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Provably strict generalisation benefit for equivariant models
Bryn Elesedy and Sheheryar Zaidi · 2021
Cited alongside, same era.
Scaling scaling laws with board games
Andy L Jones · 2021
Cited alongside, same era.
Contactnets: Learning discontinuous contact dynamics with smooth, implicit representations
Samuel Pfrommer, Mathew Halm, and Michael Posa · 2021
Cited alongside, same era.
Improved generalization bounds of group invariant/equivariant deep networks via quotient feature spaces
Akiyoshi Sannai, Masaaki Imaizumi, and Makoto Kawano · 2021
Cited alongside, same era.
MACE: Higher order equivariant message passing neural networks for fast and accurate force fields
Ilyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, and Gábor Csányi · 2022
Cited alongside, same era.
E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky · 2022
Cited alongside, same era.
A pac-bayesian generalization bound for equivariant networks
Arash Behboodi, Gabriele Cesa, and Taco S Cohen · 2022
Cited alongside, same era.
Accurate structure prediction of biomolecular interactions with AlphaFold 3
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al · 2024
Closest in time.
Explaining neural scaling laws
Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma · 2024
Closest in time.
Edgi: Equivariant diffusion for planning with embodied agents
Johann Brehmer, Joey Bose, Pim De Haan, and Taco S Cohen · 2024
Closest in time.
Pybullet, a python module for physics simulation for games, robotics and machine learning
Erwin Coumans and Yunfei Bai · 2024
Closest in time.
Euclidean, projective, conformal: Choosing a geometric algebra for equivariant transformers
Pim de Haan, Taco Cohen, and Johann Brehmer · 2024
Closest in time.
Equivariant amortized inference of poses for cryo-em
Larissa de Ruijter and Gabriele Cesa · 2024
Closest in time.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al · 2024
Closest in time.
Equivariant 3d-conditional diffusion model for molecular linker design
Ilia Igashov, Hannes Stärk, Clément Vignac, Arne Schneuing, Victor Garcia Satorras, Pascal Frossard, Max Welling, Michael Bronstein, and Bruno Correia · 2024
Closest in time.
Approximation-generalization trade-offs under (approximate) group equivariance
Mircea Petrache and Shubhendu Trivedi · 2024
Closest in time.
Compute better spent: Replacing dense layers with structured matrices
Shikai Qiu, Andres Potapczynski, Marc Finzi, Micah Goldblum, and Andrew Gordon Wilson · 2024
Closest in time.
Learning rigid-body simulators over implicit shapes for large-scale scenes and vision
Yulia Rubanova, Tatiana Lopez-Guevara, Kelsey R Allen, William F Whitney, Kimberly Stachenfeld, and Tobias Pfaff · 2024
Closest in time.
Lorentz-equivariant geometric algebra transformers for high-energy physics
Jonas Spinner, Victor Bresó, Pim de Haan, Tilman Plehn, Jesse Thaler, and Johann Brehmer · 2024
Closest in time.
Mesh neural networks for se (3)-equivariant hemodynamics estimation on the artery wall
Julian Suk, Pim de Haan, Phillip Lippe, Christoph Brune, and Jelmer M Wolterink · 2024
Closest in time.
Differentiable and learnable wireless simulation with geometric transformers
Thomas Hehn, Markus Peschl, Tribhuvanesh Orekondy, Arash Behboodi, and Johann Brehmer · 2025
Closest in time.
Equivariance is dead, long live equivariance?
Chaitanya K. Joshi · 2025
Closest in time.
Architectural highlights of AlphaFold3
Oxford Protein Informatics Group · 2025
Closest in time.
Discussion on the social media platform Twitter / X on the benefits of equivariance for conformer generation and protein folding
Twitter / X users · 2025
Closest in time.
Mattergen: a generative model for inorganic materials design
Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, Xiang Fu, Sasha Shysheya, Jonathan Crabbé, Lixin Sun, Jake Smith, et al · 2025
Closest in time.