BETANAS: BalancEd TrAining and selective drop for Neural Architecture Search

Understand

Automatic neural architecture search techniques are becoming increasingly important in machine learning area.

• Especially, weight sharing methods have shown remarkable potentials on searching good network architectures with few computational resources.
• However, existing weight sharing methods mainly suffer limitations on searching strategies: these methods either uniformly train all network paths to convergence which introduces conflicts between branches and wastes a large amount of computation on unpromising candidates, or selectively train branches with different frequency which leads to unfair evaluation and comparison among paths.
• To address these issues, we propose a novel neural architecture search method with balanced training strategy to ensure fair comparisons and a selective drop mechanism to reduce conflicts among candidate paths.