The coupling effect of Lipschitz regularization in deep neural networks

Understand

We investigate robustness of deep feed-forward neural networks when input data are subject to random uncertainties.

• More specifically, we consider regularization of the network by its Lipschitz constant and emphasize its role.
• We highlight the fact that this regularization is not only a way to control the magnitude of the weights but has also a coupling effect on the network weights accross the layers.
• We claim and show evidence on a dataset that this coupling effect brings a tradeoff between robustness and expressiveness of the network.