A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search

Understand

This survey focuses on categorizing and evaluating the methods of supernet optimization in the field of Neural Architecture Search (NAS).

• Supernet optimization involves training a single, over-parameterized network that encompasses the search space of all possible network architectures.
• The survey analyses supernet optimization methods based on their approaches to spatial and temporal optimization.
• Spatial optimization relates to optimizing the architecture and parameters of the supernet and its subnets, while temporal optimization deals with improving the efficiency of selecting architectures from the supernet.