Model Complexity of Deep Learning: A Survey

Understand

Model complexity is a fundamental problem in deep learning.

• In this paper we conduct a systematic overview of the latest studies on model complexity in deep learning.
• Model complexity of deep learning can be categorized into expressive capacity and effective model complexity.
• We review the existing studies on those two categories along four important factors, including model framework, model size, optimization process and data complexity.