Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent

Understand

Designing energy-efficient networks is of critical importance for enabling state-of-the-art deep learning in mobile and edge settings where the computation and energy budgets are highly limited.

• Recently, Liu et al.
• (2019) framed the search of efficient neural architectures into a continuous splitting process: it iteratively splits existing neurons into multiple off-springs to achieve progressive loss minimization, thus finding novel architectures by gradually growing the neural network.
• However, this method was not specifically tailored for designing energy-efficient networks, and is computationally expensive on large-scale benchmarks.