Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search

Understand

Neural architecture search (NAS) has gained immense popularity owing to its ability to automate neural architecture design.

• A number of training-free metrics are recently proposed to realize NAS without training, hence making NAS more scalable.
• Despite their competitive empirical performances, a unified theoretical understanding of these training-free metrics is lacking.
• As a consequence, (a) the relationships among these metrics are unclear, (b) there is no theoretical interpretation for their empirical performances, and (c) there may exist untapped potential in existing training-free NAS, which probably can be unveiled through a unified theoretical understanding.