Built on

A. Krizhevsky, “Learning multiple layers of features from tiny images,” 2009
2009
Earlier work this paper cites.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521, no. 7553, pp. 436–444, May 2015
2015
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in ICLR , 2015
2015
Earlier work this paper cites.
S. Teerapittayanon, B. McDanel, and H. Kung, “Branchynet: Fast inference via early exiting from deep neural networks,” in ICPR , 2016
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , vol. 30. Curran Associates, Inc., 2017
2017
Earlier work this paper cites.
H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv:1708.07747 , 2017
Original
2017
Earlier work this paper cites.
F. Dai, H. Liu, Y. Ma, J. Cao, Q. Zhao, and Y. Zhang, “Dense scale network for crowd counting,” arXiv:1906.09707 , 2019
Original
2019
Earlier work this paper cites.
G. Hacohen and D. Weinshall, “On the power of curriculum learning in training deep networks,” in ICML , 2019
2019
Earlier work this paper cites.
S. Scardapane, M. Scarpiniti, E. Baccarelli, and A. Uncini, “Why should we add early exits to neural networks?” Cognitive Computation , vol. 12, no. 5, pp. 954–966, Sep 2020
2020
Earlier work this paper cites.

Similar

T. Hu, T. Chen, H. Wang, and Z. Wang, “Triple wins: Boosting accuracy, robustness and efficiency together by enabling input-adaptive inference,” in ICLR , 2020
2020
Cited alongside, same era.
G. Gao, J. Gao, Q. Liu, Q. Wang, and Y. Wang, “Cnn-based density estimation and crowd counting: A survey,” arXiv:2003.12783 , 2020
Original
2020
Cited alongside, same era.
D. Hu, L. Mou, Q. Wang, J. Gao, Y. Hua, D. Dou, and X. X. Zhu, “Ambient sound helps: Audiovisual crowd counting in extreme conditions,” arXiv:2005.07097 , 2020
Original
2020
Cited alongside, same era.
Y. Han, G. Huang, S. Song, L. Yang, H. Wang, and Y. Wang, “Dynamic neural networks: A survey,” arXiv:2102.04906 , 2021
Original
2021
Cited alongside, same era.
A. Sabet, J. Hare, B. Al-Hashimi, and G. V. Merrett, “Temporal early exits for efficient video object detection,” arXiv:2106.11208 , 2021

Then

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2021

Multi-Exit Vision Transformer for Dynamic Inference

Bakhtiarnia, Arian, Zhang, Qi, Iosifidis, Alexandros

Understand

Deep neural networks can be converted to multi-exit architectures by inserting early exit branches after some of their intermediate layers.

This allows their inference process to become dynamic, which is useful for time critical IoT applications with stringent latency requirements, but with time-variant communication and computation resources.
In particular, in edge computing systems and IoT networks where the exact computation time budget is variable and not known beforehand.
Vision Transformer is a recently proposed architecture which has since found many applications across various domains of computer vision.

Built on

A. Krizhevsky, “Learning multiple layers of features from tiny images,” 2009
2009
Earlier work this paper cites.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521, no. 7553, pp. 436–444, May 2015
2015
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in ICLR , 2015
2015
Earlier work this paper cites.
S. Teerapittayanon, B. McDanel, and H. Kung, “Branchynet: Fast inference via early exiting from deep neural networks,” in ICPR , 2016
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , vol. 30. Curran Associates, Inc., 2017
2017
Earlier work this paper cites.
H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv:1708.07747 , 2017
Original
2017
Earlier work this paper cites.
F. Dai, H. Liu, Y. Ma, J. Cao, Q. Zhao, and Y. Zhang, “Dense scale network for crowd counting,” arXiv:1906.09707 , 2019
Original
2019
Earlier work this paper cites.
G. Hacohen and D. Weinshall, “On the power of curriculum learning in training deep networks,” in ICML , 2019
2019
Earlier work this paper cites.
S. Scardapane, M. Scarpiniti, E. Baccarelli, and A. Uncini, “Why should we add early exits to neural networks?” Cognitive Computation , vol. 12, no. 5, pp. 954–966, Sep 2020
2020
Earlier work this paper cites.

Similar

T. Hu, T. Chen, H. Wang, and Z. Wang, “Triple wins: Boosting accuracy, robustness and efficiency together by enabling input-adaptive inference,” in ICLR , 2020
2020
Cited alongside, same era.
G. Gao, J. Gao, Q. Liu, Q. Wang, and Y. Wang, “Cnn-based density estimation and crowd counting: A survey,” arXiv:2003.12783 , 2020
Original
2020
Cited alongside, same era.
D. Hu, L. Mou, Q. Wang, J. Gao, Y. Hua, D. Dou, and X. X. Zhu, “Ambient sound helps: Audiovisual crowd counting in extreme conditions,” arXiv:2005.07097 , 2020
Original
2020
Cited alongside, same era.
Y. Han, G. Huang, S. Song, L. Yang, H. Wang, and Y. Wang, “Dynamic neural networks: A survey,” arXiv:2102.04906 , 2021
Original
2021
Cited alongside, same era.
A. Sabet, J. Hare, B. Al-Hashimi, and G. V. Merrett, “Temporal early exits for efficient video object detection,” arXiv:2106.11208 , 2021

Then

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

2021

Cited alongside, same era.

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR , 2021

2021

Cited alongside, same era.

S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah, “Transformers in vision: A survey,” arXiv:2101.01169 , 2021

Original

2021

Cited alongside, same era.

M.-H. Guo, Z.-N. Liu, T.-J. Mu, D. Liang, R. R. Martin, and S.-M. Hu, “Can attention enable mlps to catch up with cnns?” arXiv:2105.15078 , 2021

Original

2021

Cited alongside, same era.

I. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Unterthiner, J. Yung, A. Steiner, D. Keysers, J. Uszkoreit, M. Lucic, and A. Dosovitskiy, “Mlp-mixer: An all-mlp architecture for vision,” arXiv:2105.01601 , 2021

Original