Fetching the paper…
Reading the bibliography…
Knowledge distillation has emerged as a highly effective method for bridging the representation discrepancy between large-scale models and lightweight models.
Learning multiple layers of features from tiny images
Alex Krizhevsky and Geoffrey Hinton · 2009
Earlier work this paper cites.
Torchvision the machine-vision package of torch
Sébastien Marcel and Yann Rodriguez · 2010
Earlier work this paper cites.
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, et al · 2014
Earlier work this paper cites.
Fitnets: Hints for thin deep nets
Adriana Romero, Nicolas Ballas, et al · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al · 2015
Earlier work this paper cites.
Faster r-cnn: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, et al · 2015
Earlier work this paper cites.
The cityscapes dataset for semantic urban scene understanding
Marius Cordts, Mohamed Omran, and Ramos et al · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, et al · 2016
Earlier work this paper cites.
Quantized convolutional neural networks for mobile devices
Jiaxiang Wu, Cong Leng, Yuhang Wang, Qinghao Hu, and Jian Cheng · 2016
Earlier work this paper cites.
Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer
Sergey Zagoruyko and Nikos Komodakis · 2016
Earlier work this paper cites.
Wide residual networks
Sergey Zagoruyko and Nikos Komodakis · 2016
Earlier work this paper cites.
Cascade r-cnn: Delving into high quality object detection
Zhaowei Cai and Nuno Vasconcelos · 2017
Earlier work this paper cites.
Focal loss for dense object detection
Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár · 2017
Earlier work this paper cites.
Skip connections eliminate singularities
Emin Orhan and Xaq Pitkow · 2017
Earlier work this paper cites.
Pyramid scene parsing network
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia · 2017
Earlier work this paper cites.
Variational information distillation for knowledge transfer
Sungsoo Ahn, Shell Xu Hu, et al · 2019
Earlier work this paper cites.
MMDetection: Open mmlab detection toolbox and benchmark
Kai Chen and Jiaqi Wang · 2019
Earlier work this paper cites.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
Jonathan Frankle and Michael Carbin · 2019
Earlier work this paper cites.
A comprehensive overhaul of feature distillation
Byeongho Heo, Jeesoo Kim, and Sangdoo et al · 2019
Earlier work this paper cites.
Similarity of neural network representations revisited
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton · 2019
Earlier work this paper cites.
Relational knowledge distillation
Wonpyo Park, Dongju Kim, et al · 2019
Cited alongside, same era.
Correlation congruence for knowledge distillation
Baoyun Peng, Xiao Jin, et al · 2019
Cited alongside, same era.
Fcos: Fully convolutional one-stage object detection
Zhi Tian, Chunhua Shen, Hao Chen, and Tong He · 2019
Cited alongside, same era.
Similarity-preserving knowledge distillation
Frederick Tung and Greg Mori · 2019
Cited alongside, same era.
Distilling object detectors with fine-grained feature imitation
Tao Wang, Li Yuan, Xiaopeng Zhang, and Jiashi Feng · 2019
Cited alongside, same era.
Reppoints: Point set representation for object detection
Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin · 2019
Cited alongside, same era.
Improve object detection with feature-based knowledge distillation: Towards accurate and efficient detectors
Linfeng Zhang and Kaisheng Ma · 2021
Later among the works it cites.
Distilling object detectors with feature richness
Du Zhixing, Rui Zhang, Ming Chang, Shaoli Liu, Tianshi Chen, Yunji Chen, et al · 2021
Later among the works it cites.
Bootstrap generalization ability from loss landscape perspective
Huanran Chen, Shitong Shao, Ziyi Wang, Zirui Shang, Jin Chen, Xiaofeng Ji, and Xinxiao Wu · 2022
Later among the works it cites.
Monodistill: Learning spatial features for monocular 3d object detection
Zhiyu Chong, Xinzhu Ma, Hong Zhang, Yuxin Yue, Haojie Li, Zhihui Wang, and Wanli Ouyang · 2022
Later among the works it cites.
Structural knowledge distillation for object detection
Philip de Rijk, Lukas Schneider, Marius Cordts, and Dariu M. Gavrila · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
A unifying mutual information view of metric learning: Cross-entropy vs. pairwise losses
Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Piantanida, and Ismail Ben Ayed · 2020
Cited alongside, same era.
MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark, 2020
MMSegmentation Contributors · 2020
Cited alongside, same era.
Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection
Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang, and Jian Yang · 2020
Cited alongside, same era.
Thao Nguyen, Maithra Raghu, and Simon Kornblith · 2020
Cited alongside, same era.
Contrastive representation distillation
Yonglong Tian, Dilip Krishnan, and Phillip Isola · 2020
Cited alongside, same era.
Training data-efficient image transformers & distillation through attention
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herv’e J’egou · 2020
Cited alongside, same era.
Knowledge distillation from a stronger teacher
Tao Huang, Shan You, et al · 2022
Later among the works it cites.
Better teacher better student: Dynamic prior knowledge for knowledge distillation
Zengyu Qiu, Xinzhu Ma, Kunlin Yang, Chunya Liu, Jun Hou, Shuai Yi, and Wanli Ouyang · 2022
Later among the works it cites.
Distilling representational similarity using centered kernel alignment (cka)
Aninda Saha, Alina Bialkowski, and Sara Khalifa · 2022
Later among the works it cites.
Focal and global knowledge distillation for detectors
Zhendong Yang, Zhe Li, et al · 2022
Later among the works it cites.
Masked generative distillation
Zhendong Yang, Zhe Li, Mingqi Shao, Dachuan Shi, Zehuan Yuan, and Chun Yuan · 2022
Later among the works it cites.
Vitkd: Practical guidelines for vit feature knowledge distillation
Zhendong Yang, Zhe Li, Ailing Zeng, Zexian Li, Chun Yuan, and Yu Li · 2022
Later among the works it cites.
Decoupled knowledge distillation
Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang · 2022
Later among the works it cites.
Robust classification via a single diffusion model
Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang, Chengqi Duan, Hang Su, and Jun Zhu · 2023
Later among the works it cites.
Decoupled kullback-leibler divergence loss
Jiequan Cui, Zhuotao Tian, Zhisheng Zhong, Xiaojuan Qi, Bei Yu, and Hanwang Zhang · 2023
Later among the works it cites.
Adaptive hierarchy-branch fusion for online knowledge distillation
Linrui Gong, Shaohui Lin, Baochang Zhang, Yunhang Shen, Ke Li, Ruizhi Qiao, Bohan Ren, Muqing Li, Zhou Yu, and Lizhuang Ma · 2023
Later among the works it cites.
Dual relation knowledge distillation for object detection
Zhenliang Ni, Fu-Han Yang, Shengzhao Wen, and Gang Zhang · 2023
Later among the works it cites.
Teaching what you should teach: a data-based distillation method
Shitong Shao, Huanran Chen, Zhen Huang, Linrui Gong, Shuai Wang, and Xinxiao Wu · 2023
Later among the works it cites.
Catch-up distillation: You only need to train once for accelerating sampling
Shitong Shao, Xu Dai, Shouyi Yin, Lujun Li, Huanran Chen, and Yang Hu · 2023
Later among the works it cites.
Zhendong Yang, Ailing Zeng, Zhe Li, Tianke Zhang, Chun Yuan, and Yu Li · 2023
Later among the works it cites.
Afn: Adaptive fusion normalization via an encoder-decoder framework
Zikai Zhou and Huanran Chen · 2023
Later among the works it cites.