2021

Data-Free Knowledge Transfer: A Survey

Liu, Yuang, Zhang, Wei, Wang, Jun et al.

Understand

In the last decade, many deep learning models have been well trained and made a great success in various fields of machine intelligence, especially for computer vision and natural language processing.

To better leverage the potential of these well-trained models in intra-domain or cross-domain transfer learning situations, knowledge distillation (KD) and domain adaptation (DA) are proposed and become research highlights.
They both aim to transfer useful information from a well-trained model with original training data.
However, the original data is not always available in many cases due to privacy, copyright or confidentiality.

Built on

B. Heo, J. Kim, S. Yun, H. Park, N. Kwak, and J. Y. Choi, “A comprehensive overhaul of feature distillation,” in International Conference on Computer Vision , 2019, pp. 1921–1930
1930
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998
1998
Earlier work this paper cites.
J. Hoffman, E. Tzeng, T. Park, J.-Y. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell, “CyCADA: Cycle-consistent adversarial domain adaptation,” in International Conference on Machine Learning , 2018, pp. 1989–1998
1998
Earlier work this paper cites.
C. Buciluǎ, R. Caruana, and A. Niculescu-Mizil, “Model compression,” in ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2006, pp. 535–541
2006
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems , vol. 25, 2012, pp. 1097–1105
2012
Earlier work this paper cites.
I. Kuzborskij and F. Orabona, “Stability and hypothesis transfer learning,” in International Conference on Machine Learning . PMLR, 2013, pp. 942–950
2013
Earlier work this paper cites.
M. Suk and B. Prabhakaran, “Real-time mobile facial expression recognition system-a case study,” in IEEE Conference on Computer Vision and Pattern Recognition Workshop , 2014, pp. 132–137
2014
Earlier work this paper cites.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems , vol. 27, 2014
2014
Earlier work this paper cites.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015
Original
2015
Earlier work this paper cites.
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, “FitNets: Hints for thin deep nets,” in International Conference on Learning Representations , 2015
2015
Earlier work this paper cites.
A. Mordvintsev, C. Olah, and M. Tyka, “Inceptionism: Going deeper into neural networks,” 2015
2015
Earlier work this paper cites.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2015, p. 3431–3440
2015
Earlier work this paper cites.
Y. Goldberg, “A primer on neural network models for natural language processing,” Journal of Artificial Intelligence Research , vol. 57, pp. 345–420, 2016
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
M. Treml, J. Arjona-Medina, T. Unterthiner, R. Durgesh, F. Friedmann, P. Schuberth, A. Mayr, M. Heusel, M. Hofmarcher, M. Widrich et al. , “Speeding up semantic segmentation for autonomous driving,” in Advances in Neural Information Processing Systems , 2016
2016
Earlier work this paper cites.
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2016, pp. 3213–3223
2016
Earlier work this paper cites.
B. Chidlovskii, S. Clinchant, and G. Csurka, “Domain adaptation in the absence of source domain data,” in ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2016, pp. 451–460
2016
Earlier work this paper cites.
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” The journal of machine learning research , vol. 17, no. 1, pp. 2096–2030, 2016
2016
Earlier work this paper cites.
S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2017, pp. 1492–1500
2017
Earlier work this paper cites.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 39, no. 6, p. 1137–1149, 2017
2017
Earlier work this paper cites.
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” arXiv preprint arXiv:1706.05587 , 2017
Original
2017
Earlier work this paper cites.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2017, pp. 4700–4708
2017
Earlier work this paper cites.
H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” in International Conference on Learning Representations , 2017
2017
Earlier work this paper cites.
R. G. Lopes, S. Fenu, and T. Starner, “Data-free knowledge distillation for deep neural networks,” in Advances in Neural Information Processing Systems , 2017
2017
Earlier work this paper cites.
S. Zagoruyko and N. Komodakis, “Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer,” in International Conference on Learning Representations , 2017
2017
Earlier work this paper cites.
H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “Mixup: Beyond empirical risk minimization,” in International Conference on Learning Representations , 2017
2017
Earlier work this paper cites.
K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,” IEEE Signal Processing Magazine , vol. 34, no. 6, pp. 26–38, 2017
2017
Earlier work this paper cites.
Y. Zhou, S.-M. Moosavi-Dezfooli, N.-M. Cheung, and P. Frossard, “Adaptive quantization for deep neural network,” in AAAI Conference on Artificial Intelligence , 2018
2018
Earlier work this paper cites.
A. Polino, R. Pascanu, and D. Alistarh, “Model compression via distillation and quantization,” in International Conference on Learning Representations , 2018
2018
Earlier work this paper cites.
Z. Luo, J.-T. Hsieh, L. Jiang, J. C. Niebles, and L. Fei-Fei, “Graph distillation for action detection with privileged modalities,” in European Conference on Computer Vision , 2018, pp. 166–183
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Earlier work this paper cites.
A. R. Nelakurthi, R. Maciejewski, and J. He, “Source free domain adaptation using an off-the-shelf classifier,” in IEEE International Conference on Big Data . IEEE, 2018, pp. 140–145
2018
Earlier work this paper cites.
M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” Neurocomputing , vol. 312, pp. 135–153, 2018
2018
Earlier work this paper cites.
J. Tang and K. Wang, “Ranking distillation: Learning compact ranking models with high performance for recommender system,” in ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2018, pp. 2289–2298
2018
Earlier work this paper cites.
Y. Zhang, T. Xiang, T. M. Hospedales, and H. Lu, “Deep mutual learning,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2018, pp. 4320–4328
2018
Earlier work this paper cites.
Y. Zou, Z. Yu, B. Vijaya Kumar, and J. Wang, “Unsupervised domain adaptation for semantic segmentation via class-balanced self-training,” in European Conference on Computer Vision , 2018, pp. 289–305
2018
Earlier work this paper cites.
M. Caron, P. Bojanowski, A. Joulin, and M. Douze, “Deep clustering for unsupervised learning of visual features,” in European Conference on Computer Vision , 2018, pp. 132–149
2018
Earlier work this paper cites.
J. Kim, Y. Bhalgat, J. Lee, C. Patel, and N. Kwak, “QKD: Quantization-aware knowledge distillation,” arXiv preprint arXiv:1911.12491 , 2019
Original
2019
Earlier work this paper cites.
L. Chen, C. Yu, and L. Chen, “A new knowledge distillation for incremental object detection,” in International Joint Conference on Neural Networks . IEEE, 2019, pp. 1–7
2019
Earlier work this paper cites.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al. , “Pytorch: An imperative style, high-performance deep learning library,” Advances in Neural Information Processing Systems , vol. 32, pp. 8026–8037, 2019
2019
Earlier work this paper cites.
Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick, “Detectron2,” https://github.com/facebookresearch/detectron2 , 2019
2019
Earlier work this paper cites.
K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin, “MMDetection: Open mmlab detection toolbox and benchmark,” arXiv preprint arXiv:1906.07155 , 2019
Original
2019
Earlier work this paper cites.
G. K. Nayak, K. R. Mopuri, V. Shaj, V. B. Radhakrishnan, and A. Chakraborty, “Zero-shot knowledge distillation in deep networks,” in International Conference on Machine Learning . PMLR, 2019, pp. 4743–4751
2019
Earlier work this paper cites.
H. Chen, Y. Wang, C. Xu, Z. Yang, C. Liu, B. Shi, C. Xu, C. Xu, and Q. Tian, “Data-free learning of student networks,” in International Conference on Computer Vision , 2019, pp. 3514–3522
2019
Earlier work this paper cites.
P. Micaelli and A. Storkey, “Zero-shot knowledge transfer via adversarial belief matching,” in Advances in Neural Information Processing Systems , 2019
2019
Earlier work this paper cites.
K. Bhardwaj, N. Suda, and R. Marculescu, “Dream distillation: A data-independent model compression framework,” in International Conference on Machine Learning , 2019
2019
Earlier work this paper cites.
W. M. Kouw and M. Loog, “A review of domain adaptation without target labels,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 43, no. 3, pp. 766–785, 2019
2019
Earlier work this paper cites.
Y. Liu, K. Chen, C. Liu, Z. Qin, Z. Luo, and J. Wang, “Structured knowledge distillation for semantic segmentation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 2604–2613
2019
Earlier work this paper cites.
F. Zhang, X. Zhu, and M. Ye, “Fast human pose estimation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 3517–3526
2019
Earlier work this paper cites.
J. Deng, Y. Pan, T. Yao, W. Zhou, H. Li, and T. Mei, “Relation distillation networks for video object detection,” in International Conference on Computer Vision , 2019, pp. 7023–7032
2019
Earlier work this paper cites.
C. Yang, L. Xie, C. Su, and A. L. Yuille, “Snapshot distillation: Teacher-student optimization in one generation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 2859–2868
2019
Earlier work this paper cites.
T. Wang, L. Yuan, X. Zhang, and J. Feng, “Distilling object detectors with fine-grained feature imitation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 4933–4942
2019
Earlier work this paper cites.
W. Park, D. Kim, Y. Lu, and M. Cho, “Relational knowledge distillation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 3967–3976
2019
Earlier work this paper cites.
Y. Liu, J. Cao, B. Li, C. Yuan, W. Hu, Y. Li, and Y. Duan, “Knowledge distillation via instance relationship graph,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 7096–7104
2019
Earlier work this paper cites.
Y. Hou, Z. Ma, C. Liu, and C. C. Loy, “Learning lightweight lane detection cnns by self attention distillation,” in International Conference on Computer Vision , 2019, pp. 1013–1021
2019
Earlier work this paper cites.
S. Park and N. Kwak, “FEED: Feature-level ensemble for knowledge distillation,” arXiv preprint arXiv:1909.10754 , 2019
Original
2019
Earlier work this paper cites.
Y.-H. Tsai, K. Sohn, S. Schulter, and M. Chandraker, “Domain adaptation for structured output via discriminative patch representations,” in International Conference on Computer Vision , 2019, pp. 1456–1465
2019
Earlier work this paper cites.
T.-H. Vu, H. Jain, M. Bucher, M. Cord, and P. Pérez, “Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 2517–2526
2019
Earlier work this paper cites.
Y. Li, L. Yuan, and N. Vasconcelos, “Bidirectional learning for domain adaptation of semantic segmentation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 6936–6945
2019
Earlier work this paper cites.
L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” in Advances in Neural Information Processing Systems , vol. 32, 2019
2019
Earlier work this paper cites.
J. Yoo, M. Cho, T. Kim, and U. Kang, “Knowledge extraction with no observable data,” in Advances in Neural Information Processing Systems , 2019, pp. 2701–2710
2019
Earlier work this paper cites.
M. Nagel, M. v. Baalen, T. Blankevoort, and M. Welling, “Data-free quantization through weight equalization and bias correction,” in International Conference on Computer Vision , 2019, pp. 1325–1334
2019
Earlier work this paper cites.
X. Li, S. Zhang, B. Jiang, Y. Qi, M. C. Chuah, and N. Bi, “DAC: Data-free automatic acceleration of convolutional networks,” in IEEE Winter Conference on Applications of Computer Vision , 2019, pp. 1598–1606
2019
Earlier work this paper cites.
S. Sun, Y. Cheng, Z. Gan, and J. Liu, “Patient knowledge distillation for ber model compression,” in Conference on Empirical Methods in Natural Language Processing , 2019, pp. 4323–4332
2019
Earlier work this paper cites.
G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, “Continual lifelong learning with neural networks: A review,” Neural Networks , vol. 113, pp. 54–71, 2019
2019
Earlier work this paper cites.
J. Liang, R. He, Z. Sun, and T. Tan, “Distant supervised centroid shift: A simple and efficient approach to visual domain adaptation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 2975–2984
2019
Earlier work this paper cites.
K. Han, A. Vedaldi, and A. Zisserman, “Learning to discover novel visual categories via deep transfer clustering,” in International Conference on Computer Vision , 2019, pp. 8401–8409
2019
Earlier work this paper cites.
S. Shi, X. Wang, and H. Li, “PointRCNN: 3d object proposal generation and detection from point cloud,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 770–779
2019
Earlier work this paper cites.
X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang, “Moment matching for multi-source domain adaptation,” in International Conference on Computer Vision , 2019, pp. 1406–1415
2019
Earlier work this paper cites.
X. Yu, T. Liu, M. Gong, K. Zhang, K. Batmanghelich, and D. Tao, “Label-noise robust domain adaptation,” in International Conference on Machine Learning . PMLR, 2020, pp. 10 913–10 924
2019
Earlier work this paper cites.
H. Cha, J. Park, H. Kim, M. Bennis, and S.-L. Kim, “Proxy experience replay: Federated distillation for distributed reinforcement learning,” IEEE Intelligent Systems , vol. 35, no. 4, pp. 94–101, 2020
2020
Earlier work this paper cites.

Similar

F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He, “A comprehensive survey on transfer learning,” Proceedings of the IEEE , vol. 109, no. 1, pp. 43–76, 2020
2020
Cited alongside, same era.
M. Contributors, “MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark,” https://github.com/open-mmlab/mmsegmentation , 2020
2020
Cited alongside, same era.
H. Yin, P. Molchanov, J. M. Alvarez, Z. Li, A. Mallya, D. Hoiem, N. K. Jha, and J. Kautz, “Dreaming to distill: Data-free knowledge transfer via deepinversion,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 8715–8724
2020
Cited alongside, same era.
C. Saltori, S. Lathuilière, N. Sebe, E. Ricci, and F. Galasso, “SF-UDA 3D
2020
Cited alongside, same era.
G. Fang, J. Song, C. Shen, X. Wang, D. Chen, and M. Song, “Data-free adversarial distillation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020
2020
Cited alongside, same era.
J. Liang, D. Hu, and J. Feng, “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,” in International Conference on Machine Learning . PMLR, 2020, pp. 6028–6039
2020
Cited alongside, same era.
Y. Hou and L. Zheng, “Source free domain adaptation with image translation,” arXiv preprint arXiv:2008.07514 , 2020
Original
2020
Cited alongside, same era.
Y. Wang, W. Zhou, T. Jiang, X. Bai, and Y. Xu, “Intra-class feature variation distillation for semantic segmentation,” in European Conference on Computer Vision , 2020, pp. 346–362
2020
Cited alongside, same era.
P. Weinzaepfel, R. Brégier, H. Combaluzier, V. Leroy, and G. Rogez, “DOPE: Distillation of part experts for whole-body 3d pose estimation in the wild,” in European Conference on Computer Vision . Springer, 2020, pp. 380–397
2020
Cited alongside, same era.
S. Mukherjee and A. Awadallah, “TinyMBERT: Multi-stage distillation framework for massive multi-lingual ner,” in Annual Meeting of the Association for Computational Linguistics , 2020
2020
Cited alongside, same era.
W. Liu, P. Zhou, Z. Zhao, Z. Wang, H. Deng, and Q. Ju, “FastBERT: a self-distilling bert with adaptive inference time,” in Annual Meeting of the Association for Computational Linguistics , 2020
2020
Cited alongside, same era.
Y. Zhang, Z. Lan, Y. Dai, F. Zeng, Y. Bai, J. Chang, and Y. Wei, “Prime-aware adaptive distillation,” in European Conference on Computer Vision , 2020, pp. 658–674
2020
Cited alongside, same era.

Then

P. Han, J. Park, S. Wang, and Y. Liu, “Robustness and diversity seeking data-free knowledge distillation,” in International Conference on Acoustics, Speech and Signal Processing . IEEE, 2021, pp. 2740–2744
2021
Closest in time.
S. Kariyappa, A. Prakash, and M. K. Qureshi, “MAZE: Data-free model stealing attack using zeroth-order gradient estimation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 13 814–13 823
2021
Closest in time.
T. Miura, S. Hasegawa, and T. Shibahara, “MEGEX: Data-free model extraction attack against gradient-based explainable ai,” arXiv preprint arXiv:2107.08909 , 2021
Original
2021
Closest in time.
H. Chen, T. Guo, C. Xu, W. Li, C. Xu, C. Xu, and Y. Wang, “Learning student networks in the wild,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 6428–6437
2021
Closest in time.
G. K. Nayak, K. R. Mopuri, and A. Chakraborty, “Effectiveness of arbitrary transfer sets for data-free knowledge distillation,” in IEEE Winter Conference on Applications of Computer Vision , 2021, pp. 1430–1438
2021
Closest in time.
H. Zhao, X. Sun, J. Dong, H. Yu, and H. Zhou, “Dual discriminator adversarial distillation for data-free model compression,” International Journal of Machine Learning and Cybernetics , 2021
2021
Closest in time.
K. I. Lee, S. Lee, and B. C. Song, “Zero-shot knowledge distillation using label-free adversarial perturbation with taylor approximation,” IEEE Access , vol. 9, pp. 45 454–45 461, 2021
2021
Closest in time.
X. Tang and T. Lin, “Adversarial variational knowledge distillation,” in International Conference on Artificial Neural Networks . Springer, 2021, pp. 558–569
2021
Closest in time.
X. Zhang, H. Qin, Y. Ding, R. Gong, Q. Yan, R. Tao, Y. Li, F. Yu, and X. Liu, “Diversifying sample generation for data-free quantization,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021
2021
Closest in time.
H. Qin, Y. Ding, X. Zhang, A. Li, J. Wang, X. Liu, and J. Lu, “Diverse sample generation: Pushing the limit of data-free quantization,” arXiv preprint arXiv:2109.00212 , 2021
Original
2021
Closest in time.
X. He, J. Lu, W. Xu, Q. Hu, P. Wang, and J. Cheng, “Generative zero-shot network quantization,” in IEEE Conference on Computer Vision and Pattern Recognition Workshop , 2021, pp. 3000–3011
2021
Closest in time.
Y. Zhong, M. Lin, M. Chen, K. Li, Y. Shen, F. Chao, Y. Wu, F. Huang, and R. Ji, “Fine-grained data distribution alignment for post-training quantization,” arXiv preprint arXiv:2109.04186 , 2021
Original
2021
Closest in time.
Y. Zhong, M. Lin, G. Nan, J. Liu, B. Zhang, Y. Tian, and R. Ji, “IntraQ: Learning synthetic images with intra-class heterogeneity for zero-shot network quantization,” arXiv preprint arXiv:2111.09136 , 2021
Original
2021
Closest in time.
J.-B. Truong, P. Maini, R. J. Walls, and N. Papernot, “Data-free model extraction,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 4771–4780
2021
Closest in time.
W. Wang, B. Yin, T. Yao, L. Zhang, Y. Fu, S. Ding, J. Li, F. Huang, and X. Xue, “Delving into data: Effectively substitute training for black-box attack,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 4761–4770
2021
Closest in time.
X. Dong, H. Yin, J. M. Alvarez, J. Kautz, and P. Molchanov, “Deep neural networks are surprisingly reversible: A baseline for zero-shot inversion,” arXiv preprint arXiv:2107.06304 , 2021
Original
2021
Closest in time.
Z. Wang, “Zero-shot knowledge distillation from a decision-based black-box model,” in International Conference on Machine Learning , 2021
2021
Closest in time.
Z. Yue, Z. He, H. Zeng, and J. McAuley, “Black-box attacks on sequential recommenders via data-free model extraction,” in ACM Conference on Recommender Systems , 2021
2021
Closest in time.
M. Yu and S. Sun, “FE-DaST: Fast and effective data-free substitute training for black-box adversarial attacks,” Computers & Security , p. 102555, 2021
2021
Closest in time.
X. Deng and Z. Zhang, “Graph-free knowledge distillation for graph neural networks,” in International Joint Conference on Artificial Intelligence , 2021, pp. 2321–2327
2021
Closest in time.
Z. Hao, Y. Luo, H. Hu, J. An, and Y. Wen, “Data-free ensemble knowledge distillation for privacy-conscious multimedia model compression,” in ACM International Conference on Multimedia , ser. MM ’21, 2021, p. 1803–1811
2021
Closest in time.
G. K. Nayak, M. Keswani, S. Seshadri, and A. Chakraborty, “Beyond classification: Knowledge distillation using multi-object impressions,” in British Machine Vision Conference , 2021
2021
Closest in time.
G. Fang, Y. Bao, J. Song, X. Wang, D. Xie, C. Shen, and M. Song, “Mosaicking to distill: Knowledge distillation from out-of-domain data,” in Advances in Neural Information Processing Systems , 2021
2021
Closest in time.
J. Tang, X. Yang, X. Cheng, N. Jiang, W. Yu, and P. Zhang, “Data-free knowledge distillation with positive-unlabeled learning,” in International Conference on Neural Information Processing . Springer International Publishing, 2021, pp. 309–320
2021
Closest in time.
Y. Choi, M. El-Khamy, and J. Lee, “Dual-teacher class-incremental learning with data-free generative replay,” in IEEE Conference on Computer Vision and Pattern Recognition Workshop , 2021, pp. 3543–3552
2021
Closest in time.
J. Smith, Y.-C. Hsu, J. Balloch, Y. Shen, H. Jin, and Z. Kira, “Always be dreaming: A new approach for data-free class-incremental learning,” in International Conference on Computer Vision , 2021
2021
Closest in time.
Z. Huang, W. Hao, X. Wang, M. Tao, J. Huang, W. Liu, and X.-S. Hua, “Half-real half-fake distillation for class-incremental semantic segmentation,” arXiv preprint arXiv:2104.00875 , 2021
Original
2021
Closest in time.
J. Liang, D. Hu, Y. Wang, R. He, and J. Feng, “Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021
2021
Closest in time.
S. Tang, Y. Shi, Z. Ma, J. Li, J. Lyu, Q. Li, and J. Zhang, “Model adaptation through hypothesis transfer with gradual knowledge distillation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2021, pp. 5679–5685
2021
Closest in time.
W. Chen, L. Lin, S. Yang, D. Xie, S. Pu, Y. Zhuang, and W. Ren, “Self-supervised noisy label learning for source-free unsupervised domain adaptation,” arXiv preprint arXiv:2102.11614 , 2021
Original
2021
Closest in time.
S. Yang, Y. Wang, J. van de Weijer, L. Herranz, and S. Jui, “Generalized source-free domain adaptation,” in International Conference on Computer Vision , 2021, pp. 8978–8987
2021
Closest in time.
S. Tang, Y. Yang, Z. Ma, N. Hendrich, F. Zeng, S. S. Ge, C. Zhang, and J. Zhang, “Nearest neighborhood-based deep clustering for source data-absent unsupervised domain adaptation,” arXiv preprint arXiv:2107.12585 , 2021
Original
2021
Closest in time.
S. Yang, Y. Wang, J. van de Weijer, L. Herranz, and S. Jui, “Exploiting the intrinsic neighborhood structure for source-free domain adaptation,” in Advances in Neural Information Processing Systems , 2021
2021
Closest in time.
S. M. Ahmed, D. S. Raychaudhuri, S. Paul, S. Oymak, and A. K. Roy-Chowdhury, “Unsupervised multi-source domain adaptation without access to source data,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 10 103–10 112
2021
Closest in time.
P. Agarwal, D. P. Paudel, J.-N. Zaech, and L. Van Gool, “Unsupervised robust domain adaptation without source data,” arXiv preprint arXiv:2103.14577 , 2021
Original
2021
Closest in time.
G. Yang, H. Tang, Z. Zhong, M. Ding, L. Shao, N. Sebe, and E. Ricci, “Transformer-based source-free domain adaptation,” arXiv preprint arXiv:2105.14138 , 2021
Original
2021
Closest in time.
Y. Iwasawa and Y. Matsuo, “Test-time classifier adjustment module for model-agnostic domain generalization,” in Advances in Neural Information Processing Systems , vol. 34, 2021
2021
Closest in time.
J. Dong, Z. Fang, A. Liu, G. Sun, and T. Liu, “Confident anchor-induced multi-source free domain adaptation,” in Advances in Neural Information Processing Systems , 2021
2021
Closest in time.
J. Huang, D. Guan, A. Xiao, and S. Lu, “Model adaptation: Historical contrastive learning for unsupervised domain adaptation without source data,” in Advances in Neural Information Processing Systems , 2021
2021
Closest in time.
H. Xia, H. Zhao, and Z. Ding, “Adaptive adversarial network for source-free domain adaptation,” in International Conference on Computer Vision , 2021, pp. 9010–9019
2021
Closest in time.
Y. Du, H. Yang, M. Chen, J. Jiang, H. Luo, and C. Wang, “Generation, augmentation, and alignment: A pseudo-source domain based method for source-free domain adaptation,” arXiv preprint arXiv:2109.04015 , 2021
Original
2021
Closest in time.
H. Zhang, Y. Zhang, K. Jia, and L. Zhang, “Unsupervised domain adaptation of black-box source models,” arXiv preprint arXiv:2101.02839 , 2021
Original
2021
Closest in time.
V. K. Kurmi, V. K. Subramanian, and V. P. Namboodiri, “Domain impression: A source data free domain adaptation method,” in IEEE Winter Conference on Applications of Computer Vision , 2021, pp. 615–625
2021
Closest in time.
J. Tian, J. Zhang, W. Li, and D. Xu, “VDM-DA: Virtual domain modeling for source data-free domain adaptation,” arXiv preprint arXiv:2103.14357 , 2021
Original
2021
Closest in time.
Z. Qiu, Y. Zhang, H. Lin, S. Niu, Y. Liu, Q. Du, and M. Tan, “Source-free domain adaptation via avatar prototype generation and adaptation,” in International Joint Conference on Artificial Intelligence , 2021, pp. 2921–2927
2021
Closest in time.
A. Frikha, H. Chen, D. Krompaß, T. Runkler, and V. Tresp, “Towards data-free domain generalization,” in Advances in Neural Information Processing Systems , 2021
2021
Closest in time.
C. Eastwood, I. Mason, C. K. Williams, and B. Schölkopf, “Source-free adaptation to measurement shift via bottom-up feature restoration,” arXiv preprint arXiv:2107.05446 , 2021
Original
2021
Closest in time.
M. Ishii and M. Sugiyama, “Source-free domain adaptation via distributional alignment by matching batch normalization statistics,” arXiv preprint arXiv:2101.10842 , 2021
Original
2021
Closest in time.
J. Li, Z. Du, L. Zhu, Z. Ding, K. Lu, and H. Shen, “Divergence-agnostic unsupervised domain adaptation by adversarial attacks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021
2021
Closest in time.
F. You, J. Li, L. Zhu, K. Lu, Z. Chen, and Z. Huang, “Domain adaptive semantic segmentation without source data,” in ACM International Conference on Multimedia , 2021
2021
Closest in time.
M. Ye, J. Zhang, J. Ouyang, and D. Yuan, “Source data-free unsupervised domain adaptation for semantic segmentation,” in ACM International Conference on Multimedia , 2021, p. 2233–2242
2021
Closest in time.
F. Fleuret et al. , “Uncertainty reduction for model adaptation in semantic segmentation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 9613–9623
2021
Closest in time.
Y. Zhao, Z. Zhong, Z. Luo, G. H. Lee, and N. Sebe, “Source-free open compound domain adaptation in semantic segmentation,” in International Conference on Computer Vision , 2021
2021
Closest in time.
J. N. Kundu, A. Kulkarni, A. Singh, V. Jampani, and R. V. Babu, “Generalize then adapt: Source-free domain adaptive semantic segmentation,” in International Conference on Computer Vision , 2021
2021
Closest in time.
Y. Wang, J. Liang, and Z. Zhang, “Give me your trained model: Domain adaptive semantic segmentation without source data,” arXiv preprint arXiv:2106.11653 , 2021
Original
2021
Closest in time.
X. Luo, W. Chen, Y. Tan, C. Li, Y. He, and X. Jia, “Exploiting negative learning for implicit pseudo label rectification in source-free domain adaptive semantic segmentation,” arXiv preprint arXiv:2106.12123 , 2021
Original
2021
Closest in time.
V. Prabhu, S. Khare, D. Kartik, and J. Hoffman, “S4T: Source-free domain adaptation for semantic segmentation via self-supervised selective self-training,” arXiv preprint arXiv:2107.10140 , 2021
Original
2021
Closest in time.
F. You, J. Li, and Z. Zhao, “Test-time batch statistics calibration for covariate shift,” arXiv preprint arXiv:2110.04065 , 2021
Original
2021
Closest in time.
J. Hong, Y.-d. Zhang, and W. Chen, “Source-free unsupervised domain adaptation for cross-modality abdominal multi-organ segmentation,” arXiv preprint arXiv:2111.12221 , 2021
Original
2021
Closest in time.
V. Olsson, W. Tranheden, J. Pinto, and L. Svensson, “ClassMix: Segmentation-based data augmentation for semi-supervised learning,” in IEEE Winter Conference on Applications of Computer Vision , 2021, pp. 1369–1378
2021
Closest in time.
L. Xiong, M. Ye, D. Zhang, Y. Gan, X. Li, and Y. Zhu, “Source data-free domain adaptation of object detector through domain-specific perturbation,” International Journal of Intelligent Systems , 2021
2021
Closest in time.
D. Hegde, V. Sindagi, V. Kilic, A. B. Cooper, M. Foster, and V. Patel, “Uncertainty-aware mean teacher for source-free unsupervised domain adaptive 3d object detection,” arXiv preprint arXiv:2109.14651 , 2021
Original
2021
Closest in time.
D. Hegde and V. Patel, “Attentive prototypes for source-free unsupervised domain adaptive 3d object detection,” arXiv preprint arXiv:2111.15656 , 2021
Original
2021
Closest in time.
X. Liu and S. Zhang, “Graph consistency based mean-teaching for unsupervised domain adaptive person re-identification,” in International Joint Conference on Artificial Intelligence , 2021, pp. 874–880
2021
Closest in time.
A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon, “A survey on contrastive self-supervised learning,” Technologies , vol. 9, no. 1, p. 2, 2021
2021
Closest in time.
G. Fang, K. Mo, X. Wang, J. Song, S. Bei, H. Zhang, and M. Song, “Up to 100 × \times faster data-free knowledge distillation,” in AAAI Conference on Artificial Intelligence , 2022
2022
Closest in time.
K. Binici, N. T. Pham, T. Mitra, and K. Leman, “Preventing catastrophic forgetting and distribution mismatch in knowledge distillation via synthetic data,” in IEEE Winter Conference on Applications of Computer Vision , 2022
2022
Closest in time.
B. Yang, H.-W. Yeh, T. Harada, and P. C. Yuen, “Model-induced generalization error bound for information-theoretic representation learning in source-data-free unsupervised domain adaptation,” IEEE Transactions on Image Processing , vol. 31, pp. 419–432, 2022
2022
Closest in time.
S. Paul, A. Khurana, and G. Aggarwal, “Unsupervised adaptation of semantic segmentation models without source data,” in AAAI Conference on Artificial Intelligence , 2022
2022
Closest in time.
M. Chen, H. Xue, and D. Cai, “Domain adaptation for semantic segmentation with maximum squares loss,” in International Conference on Computer Vision , 2019, pp. 2090–2099
2099
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…