Fetching the paper…
Reading the bibliography…
Recently, zero-shot (or training-free) Neural Architecture Search (NAS) approaches have been proposed to liberate NAS from the expensive training process.
K. Hornik, M. B. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks , vol. 2, no. 5, pp. 359–366, 1989
1989
Earlier work this paper cites.
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. , vol. 13, no. 4, pp. 600–612, 2004
2004
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems , 2012
2012
Earlier work this paper cites.
S. Liu and W. Deng, “Very deep convolutional neural network based image classification using small training sample size,” in 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR) , 2015
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
2016
Earlier work this paper cites.
2016
Earlier work this paper cites.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017
2017
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
E. Real et al. , “Large-scale Evolution of Image Classifiers,” in International Conference on Machine Learning . PMLR, 2017
2017
Earlier work this paper cites.
2017
Earlier work this paper cites.
2017
Earlier work this paper cites.
M. Raghu, B. Poole, J. M. Kleinberg, S. Ganguli, and J. Sohl-Dickstein, “On the expressive power of deep neural networks,” in Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 , ser. Proceedings of Machine Learning Research, vol. 70. PMLR, 2017, pp. 2847–2854
2017
Earlier work this paper cites.
Z. Lu, H. Pu, F. Wang, Z. Hu, and L. Wang, “The expressive power of neural networks: A view from the width,” in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA , 2017, pp. 6231–6239
2017
Earlier work this paper cites.
C. Liu et al. , “Progressive Neural Architecture Search,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018
2018
Earlier work this paper cites.
2018
Earlier work this paper cites.
H. Pham, M. Guan, B. Zoph, Q. Le, and J. Dean, “Efficient Neural Architecture Search via Parameters Sharing,” in International Conference on Machine Learning . PMLR, 2018, pp. 4095–4104
2018
Earlier work this paper cites.
K. Kandasamy, W. Neiswanger, J. Schneider, B. Poczos, and E. P. Xing, “Neural architecture search with bayesian optimisation and optimal transport,” Advances in neural information processing systems , vol. 31, 2018
2018
Earlier work this paper cites.
H. Cai, T. Chen, W. Zhang, Y. Yu, and J. Wang, “Efficient architecture search by network transformation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 32, no. 1, 2018
2018
Earlier work this paper cites.
R. Luo, F. Tian, T. Qin, E. Chen, and T.-Y. Liu, “Neural architecture optimization,” Advances in neural information processing systems , vol. 31, 2018
2018
Earlier work this paper cites.
2018
Earlier work this paper cites.
O. Sharir and A. Shashua, “On the expressive power of overlapping architectures of deep learning,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018
2018
Earlier work this paper cites.
2018
Earlier work this paper cites.
A. Jacot, C. Hongler, and F. Gabriel, “Neural tangent kernel: Convergence and generalization in neural networks,” in Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , 2018
2018
Earlier work this paper cites.
T. Serra, C. Tjandraatmadja, and S. Ramalingam, “Bounding and counting linear regions of deep neural networks,” in Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018 , ser. Proceedings of Machine Learning Research, vol. 80. PMLR, 2018, pp. 4565–4573
2018
Earlier work this paper cites.
J. Lee, J. Sohl-dickstein, J. Pennington, R. Novak, S. Schoenholz, and Y. Bahri, “Deep neural networks as gaussian processes,” in International Conference on Learning Representations , 2018
2018
Earlier work this paper cites.
2018
Earlier work this paper cites.
T. Elsken, J. H. Metzen, and F. Hutter, “Neural architecture search: A survey,” The Journal of Machine Learning Research , 2019
2019
Earlier work this paper cites.
X. Gong, S. Chang, Y. Jiang, and Z. Wang, “Autogan: Neural architecture search for generative adversarial networks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 3224–3234
2019
Earlier work this paper cites.
S. Xie, H. Zheng, C. Liu, and L. Lin, “SNAS: stochastic neural architecture search,” in International Conference on Learning Representations , 2019
2019
Earlier work this paper cites.
B. Wu et al. , “Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
H. Zhou, M. Yang, J. Wang, and W. Pan, “Bayesnas: A bayesian approach for neural architecture search,” in International conference on machine learning . PMLR, 2019, pp. 7603–7613
2019
Earlier work this paper cites.
A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan et al. , “Searching for mobilenetv3,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 1314–1324
2019
Earlier work this paper cites.
M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, and Q. V. Le, “Mnasnet: Platform-aware neural architecture search for mobile,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 2820–2828
2019
Earlier work this paper cites.
H. Mao, M. Schwarzkopf, S. B. Venkatakrishnan, Z. Meng, and M. Alizadeh, “Learning Scheduling Algorithms for Data Processing Clusters,” in ACM Special Interest Group on Data Communication , 2019, pp. 270–288
2019
Earlier work this paper cites.
W.-L. Chiang et al. , “Cluster-gcn: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2019, pp. 257–266
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
X. Dong and Y. Yang, “Searching for a robust neural architecture in four gpu hours,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 1761–1770
2019
Earlier work this paper cites.
2019
Cited alongside, same era.
X. Chen, L. Xie, J. Wu, and Q. Tian, “Progressive differentiable architecture search: Bridging the depth gap between search and evaluation,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 1294–1303
2019
Cited alongside, same era.
H. Cai, L. Zhu, and S. Han, “ProxylessNAS: Direct neural architecture search on target task and hardware,” in International Conference on Learning Representations , 2019
2019
Cited alongside, same era.
2019
Cited alongside, same era.
W. Chen, X. Gong, and Z. Wang, “Neural architecture search on imagenet in four gpu hours: A theoretically inspired perspective,” in International Conference on Learning Representations , 2021
2021
Later among the works it cites.
M. S. Abdelfattah, A. Mehrotra, Ł. Dudziak, and N. D. Lane, “Zero-cost proxies for lightweight nas,” in International Conference on Learning Representations , 2021
2021
Later among the works it cites.
X. He, K. Zhao, and X. Chu, “Automl: A survey of the state-of-the-art,” Knowledge-Based Systems , vol. 212, p. 106622, 2021
2021
Later among the works it cites.
P. Ren, Y. Xiao, X. Chang, P.-Y. Huang, Z. Li, X. Chen, and X. Wang, “A comprehensive survey of neural architecture search: Challenges and solutions,” ACM Computing Surveys (CSUR) , 2021
2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2019
Cited alongside, same era.
2019
Cited alongside, same era.
S. Lin, “Generalization and expressivity for deep nets,” IEEE Trans. Neural Networks Learn. Syst. , vol. 30, no. 5, pp. 1392–1406, 2019
2019
Cited alongside, same era.
V. Nagarajan and J. Z. Kolter, “Uniform convergence may be unable to explain generalization in deep learning,” in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada , 2019, pp. 11 611–11 622
2019
Cited alongside, same era.
N. Lee, T. Ajanthan, and P. Torr, “SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY,” in International Conference on Learning Representations , 2019
2019
Cited alongside, same era.
P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz, “Importance estimation for neural network pruning,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019 . Computer Vision Foundation / IEEE, 2019, pp. 11 264–11 272
2019
Cited alongside, same era.
L. Chizat, E. Oyallon, and F. R. Bach, “On lazy training in differentiable programming,” in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada , 2019, pp. 2933–2943
2019
Cited alongside, same era.
J. Lee, L. Xiao, S. S. Schoenholz, Y. Bahri, R. Novak, J. Sohl-Dickstein, and J. Pennington, “Wide neural networks of any depth evolve as linear models under gradient descent,” in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada , 2019, pp. 8570–8581
2019
Cited alongside, same era.
2021
Later among the works it cites.
L. Xie, X. Chen, K. Bi, L. Wei, Y. Xu, L. Wang, Z. Chen, A. Xiao, J. Chang, X. Zhang, and Q. Tian, “Weight-sharing neural architecture search: A battle to shrink the optimization gap,” ACM Comput. Surv. , vol. 54, no. 9, oct 2021
2021
Later among the works it cites.
H. Benmeziane, K. El Maghraoui, H. Ouarnoughi, S. Niar, M. Wistuba, and N. Wang, “Hardware-aware neural architecture search: Survey and taxonomy,” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21 . International Joint Conferences on Artificial Intelligence Organization, 8 2021, pp. 4322–4329, survey Track
2021
Later among the works it cites.
X. Ning, C. Tang, W. Li, Z. Zhou, S. Liang, H. Yang, and Y. Wang, “Evaluating efficient performance estimators of neural architectures,” in Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , 2021, pp. 12 265–12 277
2021
Later among the works it cites.
C. White, A. Zela, R. Ru, Y. Liu, and F. Hutter, “How powerful are performance predictors in neural architecture search?” in Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , 2021, pp. 28 454–28 469
2021
Later among the works it cites.
2021
Later among the works it cites.
X. Hu, L. Chu, J. Pei, W. Liu, and J. Bian, “Model complexity of deep learning: a survey,” Knowl. Inf. Syst. , vol. 63, no. 10, pp. 2585–2619, 2021
2021
Later among the works it cites.
L. Liu, S. Zhang, Z. Kuang, A. Zhou, J. Xue, X. Wang, Y. Chen, W. Yang, Q. Liao, and W. Zhang, “Group fisher pruning for practical network compression,” in Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , ser. Proceedings of Machine Learning Research, vol. 139. PMLR, 2021, pp. 7021–7032
2021
Later among the works it cites.
V. Lopes, S. Alirezazadeh, and L. A. Alexandre, “Epe-nas: Efficient performance estimation without training for neural architecture search,” in International Conference on Artificial Neural Networks . Springer, 2021, pp. 552–563
2021
Later among the works it cites.
J. Mellor, J. Turner, A. Storkey, and E. J. Crowley, “Neural architecture search without training,” in International Conference on Machine Learning . PMLR, 2021, pp. 7588–7598
2021
Later among the works it cites.
M. Lin, P. Wang, Z. Sun, H. Chen, X. Sun, Q. Qian, H. Li, and R. Jin, “Zen-nas: A zero-shot nas for high-performance image recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 347–356
2021
Later among the works it cites.
G. Li, S. K. Mandal, Ü. Y. Ogras, and R. Marculescu, “FLASH: fast neural architecture search with hardware optimization,” ACM Trans. Embed. Comput. Syst. , vol. 20, no. 5s, pp. 63:1–63:26, 2021
2021
Later among the works it cites.
X. Dong, L. Liu, K. Musial, and B. Gabrys, “Nats-bench: Benchmarking nas algorithms for architecture topology and size,” IEEE transactions on pattern analysis and machine intelligence , 2021
2021
Later among the works it cites.
Y. Duan, X. Chen, H. Xu, Z. Chen, X. Liang, T. Zhang, and Z. Li, “Transnas-bench-101: Improving transferability and generalizability of cross-task neural architecture search,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021 . Computer Vision Foundation / IEEE, 2021
2021
Later among the works it cites.
2021
Later among the works it cites.
H. Lee, S. Lee, S. Chong, and S. J. Hwang, “Hardware-adaptive efficient latency prediction for nas via meta-learning,” in Advances in Neural Information Processing Systems , 2021
2021
Later among the works it cites.
L. L. Zhang, S. Han, J. Wei, N. Zheng, T. Cao, Y. Yang, and Y. Liu, “nn-meter: towards accurate latency prediction of deep-learning model inference on diverse edge devices,” in MobiSys ’21: The 19th Annual International Conference on Mobile Systems, Applications, and Services, Virtual Event, Wisconsin, USA, 24 June - 2 July, 2021 . ACM, 2021, pp. 81–93
2021
Later among the works it cites.
RangiLyu, “Nanodet-plus: Super fast and high accuracy lightweight anchor-free object detection model.” https://github.com/RangiLyu/nanodet , 2021
2021
Later among the works it cites.
N. Tripuraneni, B. Adlam, and J. Pennington, “Overparameterization improves robustness to covariate shift in high dimensions,” Advances in Neural Information Processing Systems , 2021
2021
Later among the works it cites.
2022
Later among the works it cites.
2022
Later among the works it cites.
2022
Later among the works it cites.
T. M. Ingolfsson, M. Vero, X. Wang, L. Lamberti, L. Benini, and M. Spallanzani, “Reducing neural architecture search spaces with training-free statistics and computational graph clustering,” in CF ’22: 19th ACM International Conference on Computing Frontiers, Turin, Italy, May 17 - 22, 2022 . ACM, 2022, pp. 213–214
2022
Later among the works it cites.
C. White, M. Khodak, R. Tu, S. Shah, S. Bubeck, and D. Dey, “A deeper look at zero-cost proxies for lightweight nas,” in ICLR Blog Track , 2022, https://iclr-blog-track.github.io/2022/03/25/zero-cost-proxies/. [Online]. Available: https://iclr-blog-track.github.io/2022/03/25/zero-cost-proxies/
2022
Later among the works it cites.
Y. Xu and H. Zhang, “Convergence of deep convolutional neural networks,” Neural Networks , vol. 153, pp. 553–563, 2022
2022
Later among the works it cites.
Z. Zhang and Z. Jia, “Gradsign: Model performance inference with theoretical insights,” in International Conference on Learning Representations , 2022
2022
Later among the works it cites.
Z. Sun, M. Lin, X. Sun, Z. Tan, H. Li, and R. Jin, “MAE-DET: revisiting maximum entropy principle in zero-shot NAS for efficient object detection,” in International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA , ser. Proceedings of Machine Learning Research, vol. 162. PMLR, 2022, pp. 20 810–20 826
2022
Later among the works it cites.
W. Chen, W. Huang, X. Du, X. Song, Z. Wang, and D. Zhou, “Auto-scaling vision transformers without training,” in International Conference on Learning Representations , 2022
2022
Later among the works it cites.
2022
Later among the works it cites.
W. Chen, W. Huang, X. Gong, B. Hanin, and Z. Wang, “Deep architecture connectivity matters for its convergence: A fine-grained analysis,” in Advances in Neural Information Processing Systems , 2022
2022
Later among the works it cites.
2022
Later among the works it cites.
N. Klyuchnikov, I. Trofimov, E. Artemova, M. Salnikov, M. Fedorov, A. Filippov, and E. Burnaev, “Nas-bench-nlp: neural architecture search benchmark for natural language processing,” IEEE Access , vol. 10, pp. 45 736–45 747, 2022
2022
Later among the works it cites.
Paper with code, “Neural architecture search on imagenet.” https://paperswithcode.com/sota/neural-architecture-search-on-imagenet , 2023
2023
Closest in time.
W. Chen, W. Huang, and Z. Wang, ““no free lunch” in neural architectures? a joint analysis of expressivity, convergence, and generalization,” in AutoML Conference 2023 , 2023
2023
Closest in time.