A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions

Baker, B., Gupta, O., Naik, N., & Raskar, R. (2016). Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167

Original

2016

Cited alongside, same era.

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770-778)

2016

Cited alongside, same era.

Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2818-2826)

2016

Cited alongside, same era.

Saxena, S., & Verbeek, J. (2016). Convolutional neural fabrics. In Advances in Neural Information Processing Systems (pp. 4053-4061)

2016

Cited alongside, same era.

Zoph, B., Yuret, D., May, J., & Knight, K. (2016). Transfer learning for low-resource neural machine translation. arXiv preprint arXiv:1604.02201

Original

2016

Cited alongside, same era.

Ha, D., Dai, A., & Le, Q. V. (2016). Hypernetworks. arXiv preprint arXiv:1609.09106

Original

2016

Cited alongside, same era.

Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016). Improved techniques for training gans. In Advances in Neural Information Processing Systems (pp. 2234-2242)

2016

Cited alongside, same era.

Klein, A., Falkner, S., Springenberg, J. T., & Hutter, F. (2016). Learning curve prediction with Bayesian neural networks

2016

Cited alongside, same era.

Chen, L. C., Collins, M., Zhu, Y., Papandreou, G., Zoph, B., Schroff, F., … & Shlens, J. (2018). Searching for efficient multi-scale architectures for dense image prediction. In Advances in Neural Information Processing Systems (pp. 8699-8710)

2018

Later among the works it cites.

Zhang, C., Ren, M., & Urtasun, R. (2018). Graph hypernetworks for neural architecture search. arXiv preprint arXiv:1810.05749

Original

2018

Later among the works it cites.

Tran, N. T., Bui, T. A., & Cheung, N. M. (2018). Dist-gan: An improved gan using distance constraints. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 370-385)

2018

Later among the works it cites.

Wang, W., Sun, Y., & Halgamuge, S. (2018). Improving mmd-gan training with repulsive loss function. arXiv preprint arXiv:1812.09916

Original

2018

Later among the works it cites.

Hoang, Q., Nguyen, T. D., Le, T., & Phung, D. (2018). MGAN: Training generative adversarial nets with multiple generators. ICLR

2018

Later among the works it cites.

Cai, H., Zhu, L., & Han, S. (2018). Proxylessnas: Direct neural architecture search on target task and hardware. arXiv preprint arXiv:1812.00332

Original

2018

Later among the works it cites.

Elsken, T., Metzen, J. H., & Hutter, F. (2018). Efficient multi-objective neural architecture search via lamarckian evolution. arXiv preprint arXiv:1804.09081

Original

2018

Later among the works it cites.

Lu, Z., Whalen, I., Boddeti, V., Dhebar, Y. D., Deb, K., Goodman, E. D., & Banzhaf, W. (2018). NSGA-NET: A multi-objective genetic algorithm for neural architecture search. ArXiv: Computer Vision and Pattern Recognition

2018

Later among the works it cites.

Wistuba, M. (2018, September). Deep learning architecture search by neuro-cell-based evolution with function-preserving mutations. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 243-258). Springer, Cham

2018

Later among the works it cites.

Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018). Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 6848-6856)

2018

Later among the works it cites.

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

Original

2018

Later among the works it cites.

Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019). Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2820-2828)

2019

Later among the works it cites.

Saikia, T., Marrakchi, Y., Zela, A., Hutter, F., & Brox, T. (2019). AutoDispNet: Improving disparity estimation with AutoML. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1812-1823)

2019

Later among the works it cites.

Cui, J., Chen, P., Li, R., Liu, S., Shen, X., & Jia, J. (2019). Fast and practical neural architecture search. In Proceedings of the IEEE International Conference on Computer Vision (pp. 6509-6518)

2019

Later among the works it cites.

Real, E., Aggarwal, A., Huang, Y., & Le, Q. V. (2019, July). Regularized evolution for image classifier architecture search. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, pp. 4780-4789)

2019

Later among the works it cites.

Chen, X., Xie, L., Wu, J., & Tian, Q. (2019). Progressive differentiable architecture search: Bridging the depth gap between search and evaluation. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1294-1303)

2019

Later among the works it cites.

Piergiovanni, A. J., Angelova, A., Toshev, A., & Ryoo, M. S. (2019). Evolving space-time neural architectures for videos. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1793-1802)

2019

Later among the works it cites.

Bashivan, P., Tensen, M., & DiCarlo, J. J. (2019). Teacher guided architecture search. In Proceedings of the IEEE International Conference on Computer Vision (pp. 5320-5329)

2019

Later among the works it cites.

Zheng, X., Ji, R., Tang, L., Zhang, B., Liu, J., & Tian, Q. (2019). Multinomial distribution learning for effective neural architecture search. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1304-1313)

2019

Later among the works it cites.

Gong, X., Chang, S., Jiang, Y., & Wang, Z. (2019). Autogan: Neural architecture search for generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3224-3234)

2019

Later among the works it cites.

Liu, C., Chen, L. C., Schroff, F., Adam, H., Hua, W., Yuille, A. L., & Fei-Fei, L. (2019). Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 82-92)

2019

Later among the works it cites.

Zhang, Y., Qiu, Z., Liu, J., Yao, T., Liu, D., & Mei, T. (2019). Customizable architecture search for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 11641-11650)

2019

Later among the works it cites.

Chang, J., Guo, Y., MENG, G., XIANG, S., & Pan, C. (2019). DATA: Differentiable ArchiTecture Approximation. In Advances in Neural Information Processing Systems (pp. 874-884)

2019

Later among the works it cites.

Ghiasi, G., Lin, T. Y., & Le, Q. V. (2019). Nas-fpn: Learning scalable feature pyramid architecture for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7036-7045)

2019

Later among the works it cites.

Jiang, Y., Hu, C., Xiao, T., Zhang, C., & Zhu, J. (2019, November). Improved differentiable architecture search for language modeling and named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 3576-3581)

2019

Later among the works it cites.

Dong, X., & Yang, Y. (2019). Searching for a robust neural architecture in four gpu hours. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1761-1770)

2019

Later among the works it cites.

Dong, X., & Yang, Y. (2019). One-shot neural architecture search via self-evaluated template network. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3681-3690)

2019

Later among the works it cites.

Ghiasi, G., Lin, T. Y., & Le, Q. V. (2019). Nas-fpn: Learning scalable feature pyramid architecture for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7036-7045)

2019

Later among the works it cites.

Wu, B., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., … & Keutzer, K. (2019). Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 10734-10742)

2019

Later among the works it cites.

Peng, J., Sun, M., Zhang, Z. X., Tan, T., & Yan, J. (2019). Efficient neural architecture transformation search in channel-level for object detection. In Advances in Neural Information Processing Systems (pp. 14290-14299)

2019

Later among the works it cites.

Li, X., Zhou, Y., Pan, Z., & Feng, J. (2019). Partial order pruning: for best speed/accuracy trade-off in neural architecture search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 9145-9153)

2019

Later among the works it cites.

Dai, X., Zhang, P., Wu, B., Yin, H., Sun, F., Wang, Y., … & Vajda, P. (2019). Chamnet: Towards efficient network design through platform-aware model adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 11398-11407)

2019

Later among the works it cites.

Fedorov, I., Adams, R. P., Mattina, M., & Whatmough, P. (2019). Sparse: Sparse architecture search for cnns on resource-constrained microcontrollers. In Advances in Neural Information Processing Systems (pp. 4978-4990)

2019

Later among the works it cites.

Nekrasov, V., Chen, H., Shen, C., & Reid, I. (2019). Fast neural architecture search of compact semantic segmentation models via auxiliary cells. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 9126-9135)

2019

Later among the works it cites.

Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., & Le, Q. V. (2019). Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 113-123)

2019

Later among the works it cites.

Dong, X., & Yang, Y. (2019). Network pruning via transformable architecture search. In Advances in Neural Information Processing Systems (pp. 759-770)

2019

Later among the works it cites.

G. Vecchio, S. Palazzo, D. Giordano, F. Rundo and C. Spampinato, "MASK-RL: Multiagent video object segmentation framework through reinforcement learning," in IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2019.2963282

2019

Later among the works it cites.

K. Shih, C. Chiu, J. Lin and Y. Bu, "Real-time object detection with reduced-region proposal network via multi-feature concatenation," in IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 6, pp. 2164-2173, June 2020, doi: 10.1109/TNNLS.2019.2929059

2019

Later among the works it cites.

Y. Zhou, G. G. Yen and Z. Yi, "Evolutionary compression of deep neural networks for biomedical image segmentation," in IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2019.2933879

2019

Later among the works it cites.

B. Zhang, D. Xiong, J. Xie and J. Su, "Neural machine translation with GRU-gated attention model," in IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2019.2957276

2019

Later among the works it cites.

Zhang, M., Li, H., Pan, S., Chang, X., & Su, S. (2020) Overcoming multi-model forgetting in one-shot NAS with diversity maximization. In Advances in Neural Information Processing Systems

2020

Closest in time.

Cheng, X., Zhong Y., Harandi M., Dai Y., Chang, X., Li H., Drummond T., and Ge, Z. (2020) Hierarchical Neural Architecture Search for Deep Stereo Matching. In Advances in Neural Information Processing Systems

2020

Closest in time.

Zela, A., Elsken, T., Saikia, T., Marrakchi, Y., Brox, T., & Hutter, F. (2020). Understanding and robustifying differentiable architecture search.ICLR

2020

Closest in time.

Y. Zhu et al., "Deep subdomain adaptation network for image classification," in IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2020.2988928

2020

Closest in time.

Zhang, M., Li, H., Pan S., Chang X., Zhou C., Ge Z., and Su S. (2020) In IEEE Trans. Pattern Anal. Mach. Intell. (T-PAMI)

2020

Closest in time.

Sun, W., Huang, Z., Liang, M., Shao, T., & Bi, H. (2020). Cocoon image segmentation method based on fully convolutional networks. In Proceedings of the Seventh Asia International Symposium on Mechatronics (pp. 832-843). Springer, Singapore

2020

Closest in time.

Z. Ji, Y. Zhao, Y. Pang, X. Li and J. Han, "Deep attentive video summarization with distribution consistency learning," in IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2020.2991083

2020

Closest in time.

D. Zhang, J. Han, L. Zhao and T. Zhao, "From discriminant to complete: Reinforcement searching-agent learning for weakly supervised object detection," in IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2020.2969483

2020

Closest in time.

Kandasamy, K., Neiswanger, W., Schneider, J., Poczos, B., & Xing, E. P. (2018). Neural architecture search with bayesian optimisation and optimal transport. In Advances in Neural Information Processing Systems (pp. 2016-2025)

2025

Closest in time.