Built on

Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Fitnets: Hints for thin deep nets
Original
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio · 2014
Earlier work this paper cites.
Net2net: Accelerating learning via knowledge transfer
Original
Tianqi Chen, Ian Goodfellow, and Jonathon Shlens · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Original
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Original
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Actor-mimic: Deep multitask and transfer reinforcement learning
Original
Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdinov · 2015
Earlier work this paper cites.
Faster r-cnn: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun · 2015
Earlier work this paper cites.
Openai gym, 2016
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.

Similar

Sequence-level knowledge distillation
Original
Yoon Kim and Alexander M Rush · 2016
Cited alongside, same era.
Weight normalization: A simple reparameterization to accelerate training of deep neural networks
Tim Salimans and Diederik P Kingma · 2016
Cited alongside, same era.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Original
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam · 2017
Cited alongside, same era.
Multi-scale dense networks for resource efficient image classification
Original
Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Q Weinberger · 2017
Cited alongside, same era.
Dynamic deep neural networks: Optimizing accuracy-efficiency trade-offs by selective execution

Then

Shufflenet: An extremely efficient convolutional neural network for mobile devices
Original
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun · 2017
Later among the works it cites.
Label refinery: Improving imagenet classification through label progression
Original
Hessam Bagherinezhad, Maxwell Horton, Mohammad Rastegari, and Ali Farhadi · 2018
Later among the works it cites.
Data-driven sparse structure selection for deep neural networks
Zehao Huang and Naiyan Wang · 2018
Later among the works it cites.
Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation
Original
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen · 2018
Later among the works it cites.
Efficient video object segmentation via network modulation

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2019

Universally Slimmable Networks and Improved Training Techniques

Yu, Jiahui, Huang, Thomas

Understand

Slimmable networks are a family of neural networks that can instantly adjust the runtime width.

The width can be chosen from a predefined widths set to adaptively optimize accuracy-efficiency trade-offs at runtime.
In this work, we propose a systematic approach to train universally slimmable networks (US-Nets), extending slimmable networks to execute at arbitrary width, and generalizing to networks both with and without batch normalization layers.
We further propose two improved training techniques for US-Nets, named the sandwich rule and inplace distillation, to enhance training process and boost testing accuracy.

Built on

Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Fitnets: Hints for thin deep nets
Original
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio · 2014
Earlier work this paper cites.
Net2net: Accelerating learning via knowledge transfer
Original
Tianqi Chen, Ian Goodfellow, and Jonathon Shlens · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Original
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Original
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Actor-mimic: Deep multitask and transfer reinforcement learning
Original
Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdinov · 2015
Earlier work this paper cites.
Faster r-cnn: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun · 2015
Earlier work this paper cites.
Openai gym, 2016
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.

Similar

Sequence-level knowledge distillation
Original
Yoon Kim and Alexander M Rush · 2016
Cited alongside, same era.
Weight normalization: A simple reparameterization to accelerate training of deep neural networks
Tim Salimans and Diederik P Kingma · 2016
Cited alongside, same era.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Original
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam · 2017
Cited alongside, same era.
Multi-scale dense networks for resource efficient image classification
Original
Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Q Weinberger · 2017
Cited alongside, same era.
Dynamic deep neural networks: Optimizing accuracy-efficiency trade-offs by selective execution

Then

Shufflenet: An extremely efficient convolutional neural network for mobile devices
Original
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun · 2017
Later among the works it cites.
Label refinery: Improving imagenet classification through label progression
Original
Hessam Bagherinezhad, Maxwell Horton, Mohammad Rastegari, and Ali Farhadi · 2018
Later among the works it cites.
Data-driven sparse structure selection for deep neural networks
Zehao Huang and Naiyan Wang · 2018
Later among the works it cites.
Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation
Original
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen · 2018
Later among the works it cites.
Efficient video object segmentation via network modulation

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Lanlan Liu and Jia Deng · 2017

Cited alongside, same era.

Proximal policy optimization algorithms

Original

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov · 2017

Cited alongside, same era.

Mastering the game of go without human knowledge

David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al · 2017

Cited alongside, same era.

Ntire 2017 challenge on single image super-resolution: Methods and results

Radu Timofte, Eirikur Agustsson, Luc Van Gool, Ming-Hsuan Yang, Lei Zhang, Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, Kyoung Mu Lee, et al · 2017

Cited alongside, same era.

Alphastar: Mastering the real-time strategy game starcraft ii

Cited in the paper.

Original

Linjie Yang, Yanran Wang, Xuehan Xiong, Jianchao Yang, and Aggelos K Katsaggelos · 2018

Later among the works it cites.

Wide activation for efficient and accurate image super-resolution

Original

Jiahui Yu, Yuchen Fan, Jianchao Yang, Ning Xu, Xinchao Wang, and Thomas S Huang · 2018

Later among the works it cites.

Slimmable neural networks

Original

Jiahui Yu, Linjie Yang, Ning Xu, Jianchao Yang, and Thomas Huang · 2018

Later among the works it cites.

Network slimming by slimmable networks: Towards one-shot architecture search for channel numbers

Original

Jiahui Yu and Thomas Huang · 2019

Closest in time.