Fetching the paper…
Reading the bibliography…
Knowledge Distillation (KD) is a typical method for training a lightweight student model with the help of a well-trained teacher model.
Dream distillation: A data-independent model compression framework
Kartikeya Bhardwaj, Naveen Suda, and Radu Marculescu · 1905
Earlier work this paper cites.
Feature selection using a multilayer perceptron
Dennis W Ruck, Steven K Rogers, and Matthew Kabrisky · 1990
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
Reading digits in natural images with unsupervised feature learning
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng · 2011
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton · 2012
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Ya Le and Xuan Yang · 2015
Earlier work this paper cites.
Fitnets: Hints for thin deep nets
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Knowledge distillation for bilingual dictionary induction
Ndapandula Nakashole and Raphael Flauger · 2017
Earlier work this paper cites.
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms
Han Xiao, Kashif Rasul, and Roland Vollgraf · 2017
Cited alongside, same era.
Learning to specialize with knowledge distillation for visual question answering
Jonghwan Mun, Kimin Lee, Jinwoo Shin, and Bohyung Han · 2018
Cited alongside, same era.
Data-free learning of student networks
Hanting Chen, Yunhe Wang, Chang Xu, Zhaohui Yang, Chuanjian Liu, Boxin Shi, Chunjing Xu, Chao Xu, and Qi Tian · 2019
Cited alongside, same era.
A style-based generator architecture for generative adversarial networks
Tero Karras, Samuli Laine, and Timo Aila · 2019
Cited alongside, same era.
Zero-shot knowledge transfer via adversarial belief matching
Paul Micaelli and Amos J. Storkey · 2019
Cited alongside, same era.
Knowledge extraction with no observable data
Dreaming to distill: Data-free knowledge transfer via deepinversion
Hongxu Yin, Pavlo Molchanov, Jose M. Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K. Jha, and Jan Kautz · 2020
Later among the works it cites.
More grounded image captioning by distilling image-text matching model
Yuanen Zhou, Meng Wang, Daqing Liu, Zhenzhen Hu, and Hanwang Zhang · 2020
Later among the works it cites.
Knowledge distillation: A survey
Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao · 2021
Later among the works it cites.
MAZE: data-free model stealing attack using zeroth-order gradient estimation
Sanjay Kariyappa, Atul Prakash, and Moinuddin K. Qureshi · 2021
Later among the works it cites.
Bidirectional distillation for top-k recommender system
Wonbin Kweon, SeongKu Kang, and Hwanjo Yu · 2021
Later among the works it cites.
Anti-backdoor learning: Training clean models on poisoned data
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jaemin Yoo, Minyong Cho, Taebum Kim, and U Kang · 2019
Cited alongside, same era.
The knowledge within: Methods for data-free model compression
Matan Haroush, Itay Hubara, Elad Hoffer, and Daniel Soudry · 2020
Cited alongside, same era.
Inter-region affinity distillation for road marking segmentation
Yuenan Hou, Zheng Ma, Chunxiao Liu, Tak-Wai Hui, and Chen Change Loy · 2020
Cited alongside, same era.
De-rrd: A knowledge distillation framework for recommender system
SeongKu Kang, Junyoung Hwang, Wonbin Kweon, and Hwanjo Yu · 2020
Cited alongside, same era.
Interpretable foreground object search as knowledge distillation
Boren Li, Po-Yu Zhuang, Jian Gu, Mingyang Li, and Ping Tan · 2020
Cited alongside, same era.
Heterogeneous knowledge distillation using information flow modeling
Nikolaos Passalis, Maria Tzelepi, and Anastasios Tefas · 2020
Cited alongside, same era.
Generative low-bitwidth data free quantization
Shoukai Xu, Haokun Li, Bohan Zhuang, Jing Liu, Jiezhang Cao, Chuangrun Liang, and Mingkui Tan · 2020
Cited alongside, same era.
Later among the works it cites.
UMEC: unified model and embedding compression for efficient recommendation systems
Jiayi Shen, Haotao Wang, Shupeng Gui, Jianchao Tan, Zhangyang Wang, and Ji Liu · 2021
Later among the works it cites.
Privileged graph distillation for cold start recommendation
Shuai Wang, Kun Zhang, Le Wu, Haiping Ma, Richang Hong, and Meng Wang · 2021
Later among the works it cites.
Delving into data: Effectively substitute training for black-box attack
Wenxuan Wang, Bangjie Yin, Taiping Yao, Li Zhang, Yanwei Fu, Shouhong Ding, Jilin Li, Feiyue Huang, and Xiangyang Xue · 2021
Later among the works it cites.
Zero-shot knowledge distillation from a decision-based black-box model
Zi Wang · 2021
Later among the works it cites.
Fe-dast: Fast and effective data-free substitute training for black-box adversarial attacks
Mengran Yu and Shiliang Sun · 2021
Later among the works it cites.
Student surpasses teacher: Imitation attack for black-box nlp apis
Qiongkai Xu, Xuanli He, Lingjuan Lyu, Lizhen Qu, and Gholamreza Haffari · 2022
Closest in time.
Towards efficient data free black-box adversarial attack
Jie Zhang, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, Lei Zhang, and Chao Wu · 2022
Closest in time.