Fetching the paper…
Reading the bibliography…
Data-Free Knowledge Distillation (DFKD) has made significant recent strides by transferring knowledge from a teacher neural network to a student neural network without accessing the original data.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
Efficient estimation of word representations in vector space
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean · 2013
Earlier work this paper cites.
Distributed representations of sentences and documents
Quoc Le and Tomas Mikolov · 2014
Earlier work this paper cites.
Conditional generative adversarial nets
Mehdi Mirza and Simon Osindero · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Ya Le and Xuan Yang · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Sergey Zagoruyko and Nikos Komodakis · 2016
Earlier work this paper cites.
Conditional image synthesis with auxiliary classifier gans
Augustus Odena, Christopher Olah, and Jonathon Shlens · 2017
Earlier work this paper cites.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
Jonathan Frankle and Michael Carbin · 2018
Earlier work this paper cites.
Word embedding for understanding natural language: a survey
Yang Li and Tao Yang · 2018
Earlier work this paper cites.
Conditional adversarial domain adaptation
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan · 2018
Earlier work this paper cites.
Zero-shot knowledge transfer via adversarial belief matching
Paul Micaelli and Amos J Storkey · 2019
Earlier work this paper cites.
Zero-shot knowledge distillation in deep networks
Gaurav Kumar Nayak, Konda Reddy Mopuri, Vaisakh Shaj, Venkatesh Babu Radhakrishnan, and Anirban Chakraborty · 2019
Cited alongside, same era.
Sentence-bert: Sentence embeddings using siamese bert-networks
Nils Reimers and Iryna Gurevych · 2019
Cited alongside, same era.
Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf · 2019
Cited alongside, same era.
Knowledge extraction with no observable data
Jaemin Yoo, Minyong Cho, Taebum Kim, and U Kang · 2019
Cited alongside, same era.
Zeroq: A novel zero shot quantization framework
Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W Mahoney, and Kurt Keutzer · 2020
Cited alongside, same era.
Momentum adversarial distillation: Handling large distribution shifts in data-free knowledge distillation
Kien Do, Thai Hung Le, Dung Nguyen, Dang Nguyen, Haripriya Harikumar, Truyen Tran, Santu Rana, and Svetha Venkatesh · 2022
Later among the works it cites.
Up to 100x faster data-free knowledge distillation
Gongfan Fang, Kanya Mo, Xinchao Wang, Jie Song, Shitao Bei, Haofei Zhang, and Mingli Song · 2022
Later among the works it cites.
Cycle class consistency with distributional optimal transport and knowledge distillation for unsupervised domain adaptation
Tuan Nguyen, Van Nguyen, Trung Le, He Zhao, Quan Hung Tran, and Dinh Phung · 2022
Later among the works it cites.
Better teacher better student: Dynamic prior knowledge for knowledge distillation
Zengyu Qiu, Xinzhu Ma, Kunlin Yang, Chunya Liu, Jun Hou, Shuai Yi, and Wanli Ouyang · 2022
Later among the works it cites.
Focal and global knowledge distillation for detectors
Zhendong Yang, Zhe Li, Xiaohu Jiang, Yuan Gong, Zehuan Yuan, Danpei Zhao, and Chun Yuan · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Data-free network quantization with adversarial knowledge distillation
Yoojin Choi, Jihwan Choi, Mostafa El-Khamy, and Jungwon Lee · 2020
Cited alongside, same era.
Large-scale generative data-free distillation
Liangchen Luo, Mark Sandler, Zi Lin, Andrey Zhmoginov, and Andrew Howard · 2020
Cited alongside, same era.
Dreaming to distill: Data-free knowledge transfer via deepinversion
Hongxu Yin, Pavlo Molchanov, Jose M Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K Jha, and Jan Kautz · 2020
Cited alongside, same era.
Contrastive model inversion for data-free knowledge distillation
Gongfan Fang, Jie Song, Xinchao Wang, Chengchao Shen, Xingen Wang, and Mingli Song · 2021
Cited alongside, same era.
Zero-shot adversarial quantization
Yuang Liu, Wei Zhang, and Jun Wang · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Cited alongside, same era.
Robust and resource-efficient data-free knowledge distillation by generative pseudo replay
Kuluhan Binici, Shivam Aggarwal, Nam Trung Pham, Karianto Leman, and Tulika Mitra · 2022
Cited alongside, same era.
Later among the works it cites.
Decoupled knowledge distillation
Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang · 2022
Later among the works it cites.
Vulexplainer: A transformer-based hierarchical distillation for explaining vulnerability types
Michael Fu, Van Nguyen, Chakkrit Kla Tantithamthavorn, Trung Le, and Dinh Phung · 2023
Closest in time.
Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig · 2023
Closest in time.
Adversarial local distribution regularization for knowledge distillation
Thanh Nguyen-Duc, Trung Le, He Zhao, Jianfei Cai, and Dinh Phung · 2023
Closest in time.
Learning to retain while acquiring: Combating distribution-shift in adversarial data-free knowledge distillation
Gaurav Patel, Konda Reddy Mopuri, and Qiang Qiu · 2023
Closest in time.
Data-free knowledge distillation via feature exchange and activation region constraint
Shikang Yu, Jiachen Chen, Hu Han, and Shuqiang Jiang · 2023
Closest in time.
Frequency attention for knowledge distillation
Cuong Pham, Van-Anh Nguyen, Trung Le, Dinh Phung, Gustavo Carneiro, and Thanh-Toan Do · 2024
Closest in time.
Text-enhanced data-free approach for federated class-incremental learning
Minh-Tuan Tran, Trung Le, Xuan-May Le, Mehrtash Harandi, and Dinh Phung · 2024
Closest in time.