2021

Source-free Domain Adaptation via Distributional Alignment by Matching Batch Normalization Statistics

Ishii, Masato, Sugiyama, Masashi

Understand

In this paper, we propose a novel domain adaptation method for the source-free setting.

In this setting, we cannot access source data during adaptation, while unlabeled target data and a model pretrained with source data are given.
Due to lack of source data, we cannot directly match the data distributions between domains unlike typical domain adaptation algorithms.
To cope with this problem, we propose utilizing batch normalization statistics stored in the pretrained model to approximate the distribution of unobserved source data.

Handwritten digit recognition with a back-propagation network
Yann LeCun, Bernhard E Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne E Hubbard, and Lawrence D Jackel · 1990
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
The Elements of Statistical Learning — Data Mining, Inference, and Prediction
Trevor Hastie, Robert Tibshirani, and Jerome Friedman · 2009
Earlier work this paper cites.
Dataset shift in machine learning
Joaquin Quionero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence · 2009
Earlier work this paper cites.
A theory of learning from different domains
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan · 2010
Earlier work this paper cites.
Adapting visual category models to new domains
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell · 2010
Earlier work this paper cites.
Information theory: coding theorems for discrete memoryless systems
Imre Csiszar and János Körner · 2011
Earlier work this paper cites.
Reading digits in natural images with unsupervised feature learning
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng · 2011
Earlier work this paper cites.
Conditional generative adversarial nets
Original
Mehdi Mirza and Simon Osindero · 2014
Earlier work this paper cites.

Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Cited alongside, same era.
Learning transferable features with deep adaptation networks
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan · 2015
Cited alongside, same era.
Domain separation networks
Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan · 2016
Cited alongside, same era.
Domain-adversarial training of neural networks
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky · 2016
Cited alongside, same era.
Domain Adaptation in Computer Vision Applications
Gabriela Csurka (ed.) · 2017
Cited alongside, same era.
Deep transfer learning with joint adaptation networks
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan · 2017
Cited alongside, same era.
Adversarial discriminative domain adaptation
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell · 2017
Cited alongside, same era.
Adaptive batch normalization for practical domain adaptation
Yanghao Li, Naiyan Wang, Jianping Shi, Xiaodi Hou, and Jiaying Liu · 2018
Cited alongside, same era.
Conditional adversarial domain adaptation
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan · 2018
Cited alongside, same era.

Domain-specific batch normalization for unsupervised domain adaptation
Woong-Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han · 2019
Later among the works it cites.
Cluster alignment with a teacher for unsupervised domain adaptation
Zhijie Deng, Yucen Luo, and Jun Zhu · 2019
Later among the works it cites.
Transferable normalization: Towards improving transferability of deep neural networks
Ximei Wang, Ying Jin, Mingsheng Long, Jianmin Wang, and Michael I Jordan · 2019
Later among the works it cites.
d-sne: Domain adaptation using stochastic neighborhood embedding
Xiang Xu, Xiong Zhou, Ragav Venkatesan, Gurumurthy Swaminathan, and Orchid Majumder · 2019
Later among the works it cites.
Universal source-free domain adaptation
Jogendra Nath Kundu, Naveen Venkat, R Venkatesh Babu, et al · 2020
Later among the works it cites.
Model adaptation: Unsupervised domain adaptation without source data
Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu · 2020
Later among the works it cites.
Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation
Jian Liang, Dapeng Hu, and Jiashi Feng · 2020
Later among the works it cites.
A survey of unsupervised deep domain adaptation
Garrett Wilson and Diane J Cook · 2020
Later among the works it cites.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…