On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networks

Understand

Let F:R^n -> R be a feedforward ReLU neural network.

• It is well-known that for any choice of parameters, F is continuous and piecewise (affine) linear.
• We lay some foundations for a systematic investigation of how the architecture of F impacts the geometry and topology of its possible decision regions for binary classification tasks.
• Following the classical progression for smooth functions in differential topology, we first define the notion of a generic, transversal ReLU neural network and show that almost all ReLU networks are generic and transversal.