Universal approximation and model compression for radial neural networks

Understand

We introduce a class of fully-connected neural networks whose activation functions, rather than being pointwise, rescale feature vectors by a function depending only on their norm.

• We call such networks radial neural networks, extending previous work on rotation equivariant networks that considers rescaling activations in less generality.
• We prove universal approximation theorems for radial neural networks, including in the more difficult cases of bounded widths and unbounded domains.
• Our proof techniques are novel, distinct from those in the pointwise case.