On the Depth of Deep Neural Networks: A Theoretical View

Understand

People believe that depth plays an important role in success of deep neural networks (DNN).

• However, this belief lacks solid theoretical justifications as far as we know.
• We investigate role of depth from perspective of margin bound.
• In margin bound, expected error is upper bounded by empirical margin error plus Rademacher Average (RA) based capacity term.