Fetching the paper…
Reading the bibliography…
We develop an approach to efficiently grow neural networks, within which parameterization and optimization strategies are designed by considering their effects on the training dynamics.
Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu · 2002
Earlier work this paper cites.
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Adaptive subgradient methods for online learning and stochastic optimization
John C. Duchi, Elad Hazan, and Yoram Singer · 2011
Earlier work this paper cites.
ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude
Tijmen Tieleman, Geoffrey Hinton, et al · 2012
Earlier work this paper cites.
Report on the 11th IWSLT evaluation campaign
Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico · 2014
Earlier work this paper cites.
The CIFAR-10 dataset
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton · 2014
Earlier work this paper cites.
Efficient transfer learning method for automatic hyperparameter tuning
Dani Yogatama and Gideon Mann · 2014
Earlier work this paper cites.
Fast R-CNN
Ross B. Girshick · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba · 2015
Earlier work this paper cites.
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, and Trevor Darrell · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Net2net: Accelerating learning via knowledge transfer
Tianqi Chen, Ian J. Goodfellow, and Jonathon Shlens · 2016
Earlier work this paper cites.
Training and investigating residual nets
Sam Gross and Michael Wilber · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Deep networks with stochastic depth
Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q. Weinberger · 2016
Cited alongside, same era.
SSD: single shot multibox detector
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott E. Reed, Cheng-Yang Fu, and Alexander C. Berg · 2016
Cited alongside, same era.
Network morphism
Tao Wei, Changhu Wang, Yong Rui, and Chang Wen Chen · 2016
Cited alongside, same era.
Sergey Zagoruyko and Nikos Komodakis · 2016
Cited alongside, same era.
Rethinking atrous convolution for semantic image segmentation
Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam · 2017
Splitting steepest descent for growing neural architectures
Qiang Liu, Wu Lemeng, and Wang Dilin · 2019
Later among the works it cites.
Regularized evolution for image classifier architecture search
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le · 2019
Later among the works it cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2020
Later among the works it cites.
Orthogonalized SGD and nested architectures for anytime neural networks
Chengcheng Wan, Henry Hoffmann, Shan Lu, and Michael Maire · 2020
Later among the works it cites.
Autogrow: Automatic layer growing in deep convolutional networks
Wei Wen, Feng Yan, Yiran Chen, and Hai Li · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Simple and efficient architecture search for convolutional neural networks
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter · 2018
Cited alongside, same era.
Large batch training of convolutional networks with layer-wise adaptive rate scaling, 2018
Boris Ginsburg, Igor Gitman, and Yang You · 2018
Cited alongside, same era.
Scalable hyperparameter transfer learning
Valerio Perrone, Rodolphe Jenatton, Matthias W. Seeger, and Cédric Archambeau · 2018
Cited alongside, same era.
Nest: A neural network synthesis tool based on a grow-and-prune paradigm
Xiaoliang Dai, Hongxu Yin, and Niraj K. Jha · 2019
Cited alongside, same era.
Firefly neural architecture descent: a general approach for growing neural networks
Lemeng Wu, Bo Liu, Peter Stone, and Qiang Liu · 2020
Later among the works it cites.
Lemeng Wu, Mao Ye, Qi Lei, Jason D Lee, and Qiang Liu · 2020
Later among the works it cites.
Large batch optimization for deep learning: Training bert in 76 minutes
Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh · 2020
Later among the works it cites.
Hyperparameter transfer learning with adaptive complexity
Samuel Horváth, Aaron Klein, Peter Richtárik, and Cédric Archambeau · 2021
Later among the works it cites.
Domain-independent dominance of adaptive methods
Pedro Savarese, David McAllester, Sudarshan Babu, and Michael Maire · 2021
Later among the works it cites.
Tensor programs IV: feature learning in infinite-width neural networks
Greg Yang and Edward J. Hu · 2021
Later among the works it cites.
Tuning large neural networks via zero-shot hyperparameter transfer
Greg Yang, Edward J Hu, Igor Babuschkin, Szymon Sidor, Xiaodong Liu, David Farhi, Nick Ryder, Jakub Pachocki, Weizhu Chen, and Jianfeng Gao · 2021
Later among the works it cites.
Growing efficient deep networks by structured continuous sparsification
Xin Yuan, Pedro Henrique Pamplona Savarese, and Michael Maire · 2021
Later among the works it cites.
Gradmax: Growing neural networks using gradient information
Utku Evci, Bart van Merrienboer, Thomas Unterthiner, Fabian Pedregosa, and Max Vladymyrov · 2022
Later among the works it cites.
Scaling vision transformers
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer · 2022
Later among the works it cites.