Neural Inheritance Relation Guided One-Shot Layer Assignment Search

Understand

Layer assignment is seldom picked out as an independent research topic in neural architecture search.

• In this paper, for the first time, we systematically investigate the impact of different layer assignments to the network performance by building an architecture dataset of layer assignment on CIFAR-100.
• Through analyzing this dataset, we discover a neural inheritance relation among the networks with different layer assignments, that is, the optimal layer assignments for deeper networks always inherit from those for shallow networks.
• Inspired by this neural inheritance relation, we propose an efficient one-shot layer assignment search approach via inherited sampling.