AutoGrow: Automatic Layer Growing in Deep Convolutional Networks

Understand

Depth is a key component of Deep Neural Networks (DNNs), however, designing depth is heuristic and requires many human efforts.

• We propose AutoGrow to automate depth discovery in DNNs: starting from a shallow seed architecture, AutoGrow grows new layers if the growth improves the accuracy; otherwise, stops growing and thus discovers the depth.
• We propose robust growing and stopping policies to generalize to different network architectures and datasets.
• Our experiments show that by applying the same policy to different network architectures, AutoGrow can always discover near-optimal depth on various datasets of MNIST, FashionMNIST, SVHN, CIFAR10, CIFAR100 and ImageNet.