Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Nets

Understand

We derive generalization and excess risk bounds for neural nets using a family of complexity measures based on a multilevel relative entropy.

• The bounds are obtained by introducing the notion of generated hierarchical coverings of neural nets and by using the technique of chaining mutual information introduced in Asadi et al.
• NeurIPS'18.
• The resulting bounds are algorithm-dependent and exploit the multilevel structure of neural nets.