2020

BRP-NAS: Prediction-based NAS using GCNs

Dudziak, Łukasz, Chau, Thomas, Abdelfattah, Mohamed S. et al.

Understand

Neural architecture search (NAS) enables researchers to automatically explore broad design spaces in order to improve efficiency of neural networks.

This efficiency is especially important in the case of on-device deployment, where improvements in accuracy should be balanced out with computational demands of a model.
In practice, performance metrics of model are computationally expensive to obtain.
Previous work uses a proxy (e.g., number of operations) or a layer-wise measurement of neural network layers to estimate end-to-end hardware performance but the imprecise prediction diminishes the quality of NAS.

Built on

Multiobjective Optimization, Interactive and Evolutionary Approaches
Juergen Branke, Kalyan Deb, Kaisa Miettinen, and Slowinski Roman · 2008
Earlier work this paper cites.
Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves
Tobias Domhan, Jost Tobias Springenberg, and Frank Hutter · 2015
Earlier work this paper cites.
Neural architecture search with reinforcement learning
Barret Zoph and Quoc V. Le · 2017
Earlier work this paper cites.
Designing neural network architectures using reinforcement learning
Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar · 2017
Earlier work this paper cites.
Learning curve prediction with bayesian neural networks
Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, and Frank Hutter · 2017
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Thomas N. Kipf and Max Welling · 2017
Earlier work this paper cites.
Latency-Driven Design for FPGA-based Convolutional Neural Networks
S. I. Venieris and C. S. Bouganis · 2017
Earlier work this paper cites.
Learning transferable architectures for scalable image recognition
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le · 2018
Earlier work this paper cites.
Efficient architecture search by network transformation
Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, and Jun Wang · 2018
Earlier work this paper cites.
Path-level network transformation for efficient architecture search
Han Cai, Jiacheng Yang, Weinan Zhang, Song Han, and Yong Yu · 2018
Earlier work this paper cites.
Efficient neural architecture search via parameter sharing
Hieu Quang Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, and Jeff Dean · 2018
Earlier work this paper cites.

Similar

SMASH: One-shot model architecture search through hypernetworks
Andrew Brock, Theo Lim, J.M. Ritchie, and Nick Weston · 2018
Cited alongside, same era.
Progressive neural architecture search
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy · 2018
Cited alongside, same era.
Accelerating neural architecture search using performance prediction
Bowen Baker, Otkrist Gupta, Ramesh Raskar, and Nikhil Naik · 2018
Cited alongside, same era.
Neural architecture optimization
Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu · 2018
Cited alongside, same era.
DPP-Net: Device-aware progressive search for pareto-optimal neural architectures
Jin-Dong Dong, An-Chieh Cheng, Da-Cheng Juan, Wei Wei, and Min Sun · 2018
Cited alongside, same era.
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun · 2018
Cited alongside, same era.
ProxylessNAS: Direct neural architecture search on target task and hardware
Han Cai, Ligeng Zhu, and Song Han · 2019
Cited alongside, same era.
Neural predictor for neural architecture search
Original
Wei Wen, Hanxiao Liu, Hai Li, Yiran Chen, Gabriel Bender, and Pieter-Jan Kindermans · 2019
Cited alongside, same era.
How powerful are graph neural networks?
Original
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka · 2019
Cited alongside, same era.
D-VAE: A variational autoencoder for directed acyclic graphs
Muhan Zhang, Shali Jiang, Zhicheng Cui, Roman Garnett, and Yixin Chen · 2019
Cited alongside, same era.
Sample-efficient neural architecture search by learning action space
Linnan Wang, Saining Xie, Teng Li, Rodrigo Fonseca, and Yuandong Tian · 2019
Cited alongside, same era.

Then

MnasNet: Platform-aware neural architecture search for mobile
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V. Le · 2019
Later among the works it cites.
FBNet: Hardware-aware efficient convnet design via differentiable neural architecture search
Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer · 2019
Later among the works it cites.
ChamNet: Towards efficient network design through platform-aware model adaptation
Xiaoliang Dai, Peizhao Zhang, Bichen Wu, Hongxu Yin, Fei Sun, Yanghan Wang, Marat Dukhan, Yunqing Hu, Yiming Wu, Yangqing Jia, Peter Vajda, Matt Uyttendaele, and Niraj K. Jha · 2019
Later among the works it cites.
Regularized evolution for image classifier architecture search
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le · 2019
Later among the works it cites.
NAS-Bench-101: Towards reproducible neural architecture search
Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter · 2019
Later among the works it cites.
DARTS: Differentiable architecture search
Hanxiao Liu, Karen Simonyan, and Yiming Yang · 2019
Later among the works it cites.
NPENAS: Neural predictor guided evolution for neural architecture search
Chen Wei, Chuang Niu, Yiping Tang, and Jimin Liang · 2020
Closest in time.
A generic graph-based neural architecture encoding scheme for predictor-based nas
Xuefei Ning, Yin Zheng, Tianchen Zhao, Yu Wang, and Huazhong Yang · 2020
Closest in time.
Efficient sample-based neural architecture search with learnable predictor
Original
Han Shi, Renjie Pi, Hang Xu, Zhenguo Li, James T. Kwok, and Tong Zhang · 2020
Closest in time.
Approximate LSTMs for Time-Constrained Inference: Enabling Fast Reaction in Self-Driving Cars
Alexandros Kouris, Stylianos I Venieris, Michail Rizakis, and Christos-Savvas Bouganis · 2020
Closest in time.
NAS-Bench-201: Extending the scope of reproducible neural architecture search
Xuanyi Dong and Yi Yang · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…