2020

PDE-based Group Equivariant Convolutional Neural Networks

Smets, Bart, Portegies, Jim, Bekkers, Erik et al.

Understand

We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs).

In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights.
Formulating our PDEs on homogeneous spaces allows these networks to be designed with built-in symmetries such as rotation in addition to the standard translation equivariance of CNNs.
Having all the desired symmetries included in the design obviates the need to include them by means of costly techniques such as data augmentation.

Built on

“The Hamilton-Jacobi theory in the calculus of variations: its role in mathematics and physics”
Hanno Rund · 1966
Earlier work this paper cites.
“Functional Analysis”
Kosaku Yosida · 1968
Earlier work this paper cites.
“Backpropagation applied to handwritten zip code recognition”
Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard and Lawrence Jackel · 1989
Earlier work this paper cites.
“Bellman processes”
Marianne Akian, Jean-Pierre Quadrat and Michel Viot · 1994
Earlier work this paper cites.
“Integral representations of resolvents and semigroups”
Wolfgang Arendt and Alexander Bukhvalov · 1994
Earlier work this paper cites.
“Variational Methods in Image Segmentation: with Seven Image Processing Experiments”, Progress in Nonlinear Differential Equations and their Applications
J.M. Morel and S. Solimini · 1995
Earlier work this paper cites.
“Theoretical foundations of anisotropic diffusion in image processing.”
J. Weickert · 1996
Earlier work this paper cites.
“Weighted subcoercive operators on Lie groups”
AFM Ter and Derek Robinson · 1998
Earlier work this paper cites.
“Estimations du Noyau de la Chaleur sur les Espaces Homogenes”
P.Maheux · 1998
Earlier work this paper cites.
“Level Set Methods and Fast Marching Methods”
J.A. Sethian · 1999
Earlier work this paper cites.
“An operational calculus for the Euclidean motion group with applications in robotics and polymer science”
Gregory Chirikjian and Alexander Kyatkin · 2000
Earlier work this paper cites.
“Coherent states, wavelets and their generalizations”
Syed Ali, Jean-Pierre Antoine and Jean-Pierre Gazeau · 2000
Earlier work this paper cites.
“Geometric Partial Differential Equations and Image Analysis”
G. Sapiro · 2001
Earlier work this paper cites.
“A version of the Hopf-Lax formula in the Heisenberg group”
Juan Manfredi and Bianca Stroffolini · 2002
Earlier work this paper cites.
“The neurogeometry of pinwheels as a sub-Riemannian contact structure”
J. Petitot · 2003
Earlier work this paper cites.
“Morphological operations on matrix-valued images”
Bernhard Burgeth, Martin Welk, Christian Feddern and Joachim Weickert · 2004
Earlier work this paper cites.
“Ridge-based vessel segmentation in color images of the retina”
Joes Staal, Michael Abr\‘amoff, Meindert Niemeijer, Max Viergever and Bram Van · 2004
Earlier work this paper cites.
“Abstract harmonic analysis of continuous wavelet transforms”
Hartmut F\"uhr · 2005
Earlier work this paper cites.
“Nonsmooth analysis and Hamilton–Jacobi equations on Riemannian manifolds”
Daniel Azagra, Juan Ferrera and Fernando L\’opez-Mesas · 2005
Earlier work this paper cites.
“A cortical based model of perceptional completion in the roto-translation space”
G. Citti and A. Sarti · 2006
Earlier work this paper cites.
“Channel Smoothing: Efficient robust smoothing of low-level signal features”
M. Felsberg, P-E. Forssen and H. Scharr · 2006
Earlier work this paper cites.
“An efficient method for tensor voting using steerable filters”
Erik Franken, Markus van Almsick, Peter Rongen, Luc Florack and Bart ter Haar · 2006
Earlier work this paper cites.
“Scale spaces on Lie groups”
Remco Duits and Bernhard Burgeth · 2007
Earlier work this paper cites.
“Invertible orientation scores as an application of generalized wavelet theory”
Remco Duits, Maurice Duits, Markus van Almsick and Bart ter Haar · 2007
Earlier work this paper cites.
“Metric Hopf-Lax formula with semicontinuous data”
Federica Dragoni · 2007
Earlier work this paper cites.
“Group integration techniques in pattern analysis–A kernel view”, 2008
Marco Reisert · 2008
Earlier work this paper cites.
“3D stochastic completion fields for fiber tractography”
P. Momayyez-Siahkal and K. Siddiqi · 2009
Earlier work this paper cites.
“An introduction to the geometry of homogeneous spaces”
Takashi Koda · 2009
Earlier work this paper cites.
“Manifolds and differential geometry”
Jeffrey Lee, Bennett Chow, Sun-Chin Chu, David Glickenstein, Christine Guenther, James Isenberg, Tom Ivey, Dan Knopf, Peng Lu and Feng Luo · 2009
Earlier work this paper cites.
“Heat kernels on metric spaces with doubling measure”
Alexander Grigor’yan, Jiaxin Hu and Ka-Sing Lau · 2009
Earlier work this paper cites.
“Geodesic Methods in Computer Vision and Graphics”
G. Peyr\’e, M. P\’echaud, R. Keriven and L.. Cohen · 2010
Earlier work this paper cites.
“Left Invariant Parabolic Evolution Equations on SE(2) and Contour Enhancement via Invertible Orientation Scores, Part I: Linear Left-Invariant Diffusion Equations on SE(2)”
R. Duits and E.. Franken · 2010
Earlier work this paper cites.
“Left Invariant Parabolic Evolution Equations on SE(2) and Contour Enhancement via Invertible Orientation Scores, Part II: Nonlinear Left-Invariant Diffusion Equations on Invertible Orientation Scores”
R. Duits and E.. Franken · 2010
Earlier work this paper cites.
“Partial differential equations”
Lawrence Evans · 2010
Earlier work this paper cites.

Similar

“A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging”
A. Chambolle and T. Pock · 2011
Cited alongside, same era.
“Heart wall myofibers are arranged in minimal surfaces to optimize organ function”
P. Savadjiev, G.J. Strijkers, A.J. Bakermans, E. Piuze, S.W. Zucker and K. Siddiqi · 2012
Cited alongside, same era.
“Functional inequalities and Hamilton–Jacobi equations in geodesic spaces”
Zolt\’an Balogh, Alexandre Engulatov, Lars Hunziker and Outi Maasalo · 2012
Cited alongside, same era.
“Morphological and linear scale spaces for fiber enhancement in DW-MRI”
Remco Duits, Tom Haije, Eric Creusen and Arpan Ghosh · 2013
Cited alongside, same era.
“Evolution equations on Gabor transforms and their applications”
R. Duits, H. Fuehr, B.J. Janssen, L.M.J. Florack and H.A.C. van Assen · 2013
Cited alongside, same era.
“Evolution equations on Gabor transforms and their applications”
Remco Duits, Hartmut F\"uhr, Bart Janssen, Mark Bruurmijn, Luc Florack and Hans van Assen · 2013
Cited alongside, same era.
“Introduction to Smooth Manifolds”, Graduate Texts in Mathematics
J.M. Lee · 2013
Cited alongside, same era.
“Sharpening fibers in diffusion weighted MRI via erosion”
Thomas CJ Haije, Remco Duits and Chantal Tax · 2014
Cited alongside, same era.
“Hypoelliptic Diffusion and Human Vision: A Semidiscrete New Twist”
U. Boscain, R.. Chertovskih, J.. Gauthier and A.. Remizov · 2014
Cited alongside, same era.
“A Cortical-Inspired Geometry for Contour Perception and Motion Integration”
D. Barbieri, G. Citti, G. Cocci and A. Sarti · 2014
Cited alongside, same era.
“Rigid-motion scattering for texture classification”
Original
Laurent Sifre and St\’ephane Mallat · 2014
Cited alongside, same era.
“Adam: A method for stochastic optimization”
Original
Diederik Kingma and Jimmy Ba · 2014
Cited alongside, same era.

Then

“Cubenet: Equivariance to 3D rotation and translation”
Daniel Worrall and Gabriel Brostow · 2018
Later among the works it cites.
“Roto-translation covariant convolutional networks for medical image analysis”
Original
Erik Bekkers, Maxime Lafarge, Mitko Veta, Koen Eppenhof, Josien Pluim and Remco Duits · 2018
Later among the works it cites.
“Learning steerable filters for rotation equivariant CNNs”
Maurice Weiler, Fred Hamprecht and Martin Storath · 2018
Later among the works it cites.
“On the generalization of equivariance and convolution in neural networks to the action of compact groups”
Risi Kondor and Shubhendu Trivedi · 2018
Later among the works it cites.
“Learning SO(3) equivariant representations with spherical CNNs”
Carlos Esteves, Christine Allen-Blanchette, Ameesh Makadia and Kostas Daniilidis · 2018
Later among the works it cites.
“Design and Processing of Invertible Orientation Scores of 3D Images”
M… Janssen, A.. E.. Janssen, E.. Bekkers, J.án Bescós and R. Duits · 2018
Later among the works it cites.
“A geometric model of multi-scale orientation preference maps via Gabor functions”
Emre Baspinar, Giovanna Citti and Alessandro Sarti · 2018
Later among the works it cites.
“Neural ordinary differential equations”
Tian Chen, Yulia Rubanova, Jesse Bettencourt and David Duvenaud · 2018
Later among the works it cites.
“Optimal paths for variants of the 2D and 3D Reeds–Shepp car with applications in image analysis”
Remco Duits, Stephan Meesters, J-M Mirebeau and Jorg Portegies · 2018
Later among the works it cites.
“RotNIST”
Chaitanya Baweja · 2018
Later among the works it cites.
“PDE evolutions for M-smoothers: from common myths to robust numerics”
Martin Welk and Joachim Weickert · 2019
Later among the works it cites.
“A general theory of equivariant cnns on homogeneous spaces”
Taco Cohen, Mario Geiger and Maurice Weiler · 2019
Later among the works it cites.
“Total roto-translational variation”
Antonin Chambolle and Thomas Pock · 2019
Later among the works it cites.
“Geometrical optical illusion via sub-Riemannian geodesics in the roto-translation group”
B. Franceschiello, A. Mashtakov, G. Citti and A. Sarti · 2019
Later among the works it cites.
“A Cortical-Inspired Model for Orientation-Dependent Contrast Perception: A Link with Wilson-Cowan Equations”
M. Bertalm\’io, L. Calatroni, V. Franceschi, B. Franceschiello and D. Prandi · 2019
Later among the works it cites.
“Fourier transform on the homogeneous space of 3D positions and orientations for exact solutions to linear PDEs”
Remco Duits, Erik Bekkers and Alexey Mashtakov · 2019
Later among the works it cites.
“B-Spline CNNs on Lie groups”
Erik Bekkers · 2019
Later among the works it cites.
“General E(2)-Equivariant Steerable CNNs”
Maurice Weiler and Gabriele Cesa · 2019
Later among the works it cites.
“A metric model of the visual cortex”, 2019
Noemi Montobbio · 2019
Later among the works it cites.
“KerCNNs: biologically inspired lateral connections for classification of corrupted images”
Original
Noemi Montobbio, Laurent Bonnasse-Gahot, Giovanna Citti and Alessandro Sarti · 2019
Later among the works it cites.
“Geometric Image Denoising and Machine Learning”, 2019
B.M.N. Smets · 2019
Later among the works it cites.
“Hamiltonian fast marching: a numerical solver for anisotropic and non-holonomic eikonal PDEs”
Jean-Marie Mirebeau and Jorg Portegies · 2019
Later among the works it cites.
“PyTorch: An imperative style, high-performance deep learning library”
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein and Luca Antiga · 2019
Later among the works it cites.
“Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data”
Marc Finzi, Samuel Stanton, Pavel Izmailov and Andrew Wilson · 2020
Closest in time.
“Probabilistic numeric convolutional neural networks”
Original
Marc Finzi, Roberto Bondesan and Max Welling · 2020
Closest in time.
“Deep neural networks motivated by partial differential equations”
Lars Ruthotto and Eldad Haber · 2020
Closest in time.
“Pdo-econvs: Partial differential operator based equivariant convolutions”
Zhengyang Shen, Lingshen He, Zhouchen Lin and Jinwen Ma · 2020
Closest in time.
“Equivariant Deep Learning via Morphological and Linear Scale Space PDEs on the Space of Positions and Orientations”
R.Duits, B.Smets, E.J.Bekkers and J.M.Portegies · 2021
Closest in time.
“Total Variation and Mean Curvature PDEs on the Homogeneous Space of Positions and Orientations”
Bart Smets, Jim Portegies, Etienne St-Onge and Remco Duits · 2021
Closest in time.
“A bio-inspired geometric model for sound reconstruction”
Ugo Boscain, Dario Prandi, Ludovic Sacchelli and Giuseppina Turco · 2021
Closest in time.
“Coordinate Independent Convolutional Networks–Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds”
Original
Maurice Weiler, Patrick Forr\’e, Erik Verlinde and Max Welling · 2021
Closest in time.
“Steerable Partial Differential Operators for Equivariant Neural Networks”
Original
Erik Jenner and Maurice Weiler · 2021
Closest in time.
“Translating numerical concepts for PDEs into neural architectures”
Original
Tobias Alt, Pascal Peter, Joachim Weickert and Karl Schrader · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…