2016

Deep Transfer Learning with Joint Adaptation Networks

Long, Mingsheng, Zhu, Han, Wang, Jianmin et al.

Understand

Deep networks have been successfully applied to learn transferable features for adapting models from a source domain to a different target domain.

In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domains based on a joint maximum mean discrepancy (JMMD) criterion.
Adversarial training strategy is adopted to maximize JMMD such that the distributions of the source and target domains are made more distinguishable.
Learning can be performed by stochastic gradient descent with the gradients computed by back-propagation in linear-time.

Built on

Correcting sample selection bias by unlabeled data
Huang, J., Smola, A. J., Gretton, A., Borgwardt, K. M., and Schölkopf, B · 2006
Earlier work this paper cites.
A hilbert space embedding for distributions
Smola, Alex, Gretton, Arthur, Song, Le, and Schölkopf, Bernhard · 2007
Earlier work this paper cites.
Direct importance estimation with model selection and its application to covariate shift adaptation
Sugiyama, M., Nakajima, S., Kashima, H., Buenau, P. V., and Kawanabe, M · 2008
Earlier work this paper cites.
Domain adaptation: Learning bounds and algorithms
Mansour, Y., Mohri, M., and Rostamizadeh, A · 2009
Earlier work this paper cites.
Dataset shift in machine learning
Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D · 2009
Earlier work this paper cites.
Hilbert space embeddings of conditional distributions with applications to dynamical systems
Song, L., Huang, J., Smola, A., and Fukumizu, K · 2009
Earlier work this paper cites.
Kernel choice and classifiability for rkhs embeddings of probability distributions
Sriperumbudur, B. K., Fukumizu, K., Gretton, A., Lanckriet, G., and Schölkopf, B · 2009
Earlier work this paper cites.
A theory of learning from different domains
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W · 2010
Earlier work this paper cites.
A survey on transfer learning
Pan, S. J. and Yang, Q · 2010
Earlier work this paper cites.
Adapting visual category models to new domains
Saenko, K., Kulis, B., Fritz, M., and Darrell, T · 2010
Earlier work this paper cites.
Hilbert space embeddings of hidden markov models
Song, Le, Boots, Byron, Siddiqi, Sajid M, Gordon, Geoffrey J, and Smola, Alex · 2010
Earlier work this paper cites.
Hilbert space embeddings and metrics on probability measures
Sriperumbudur, Bharath K, Gretton, Arthur, Fukumizu, Kenji, Schölkopf, Bernhard, and Lanckriet, Gert RG · 2010
Earlier work this paper cites.
Cross validation framework to choose amongst models and datasets for transfer learning
Zhong, E., Fan, W., Yang, Q., Verscheure, O., and Ren, J · 2010
Earlier work this paper cites.
Natural language processing (almost) from scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., and Kuksa, P · 2011
Earlier work this paper cites.
Domain adaptation for large-scale sentiment classification: A deep learning approach
Glorot, X., Bordes, A., and Bengio, Y · 2011
Earlier work this paper cites.

Similar

Domain adaptation for object recognition: An unsupervised approach
Gopalan, R., Li, R., and Chellappa, R · 2011
Cited alongside, same era.
Domain adaptation via transfer component analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., and Yang, Q · 2011
Cited alongside, same era.
Domain transfer multiple kernel learning
Duan, L., Tsang, I. W., and Xu, D · 2012
Cited alongside, same era.
Geodesic flow kernel for unsupervised domain adaptation
Gong, B., Shi, Y., Sha, F., and Grauman, K · 2012
Cited alongside, same era.
A kernel two-sample test
Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., and Smola, A · 2012
Cited alongside, same era.
Imagenet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., and Hinton, G. E · 2012
Cited alongside, same era.
Representation learning: A review and new perspectives
Bengio, Y., Courville, A., and Vincent, P · 2013
Cited alongside, same era.
Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation
Gong, B., Grauman, K., and Sha, F · 2013
Cited alongside, same era.
Learning and transferring mid-level image representations using convolutional neural networks
Oquab, M., Bottou, L., Laptev, I., and Sivic, J · 2013
Cited alongside, same era.
Robust low rank kernel embeddings of multivariate distributions
Song, Le and Dai, Bo · 2013
Cited alongside, same era.
Kernel embeddings of conditional distributions: A unified kernel framework for nonparametric inference in graphical models
Song, Le, Fukumizu, Kenji, and Gretton, Arthur · 2013
Cited alongside, same era.
Domain adaptation under target and conditional shift
Zhang, K., Schölkopf, B., Muandet, K., and Wang, Z · 2013
Cited alongside, same era.

Then

Deep domain confusion: Maximizing for domain invariance
Original
Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., and Darrell, T · 2014
Later among the works it cites.
Flexible transfer learning under support and model shift
Wang, X. and Schneider, J · 2014
Later among the works it cites.
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., and Lipson, H · 2014
Later among the works it cites.
Unsupervised domain adaptation by backpropagation
Ganin, Y. and Lempitsky, V · 2015
Later among the works it cites.
Learning transferable features with deep adaptation networks
Long, Mingsheng, Cao, Yue, Wang, Jianmin, and Jordan, Michael I · 2015
Later among the works it cites.
On the high dimensional power of a linear-time two sample test under mean-shift alternatives
Reddi, Sashank J, Ramdas, Aaditya, Póczos, Barnabás, Singh, Aarti, and Wasserman, Larry A · 2015
Later among the works it cites.
Very deep convolutional networks for large-scale image recognition
Original
Simonyan, K. and Zisserman, A · 2015
Later among the works it cites.
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A · 2015
Later among the works it cites.
Simultaneous deep transfer across domains and tasks
Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., and Darrell, T · 2015
Later among the works it cites.
Domain separation networks
Bousmalis, Konstantinos, Trigeorgis, George, Silberman, Nathan, Krishnan, Dilip, and Erhan, Dumitru · 2016
Closest in time.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Closest in time.
Unsupervised domain adaptation with residual transfer networks
Long, Mingsheng, Zhu, Han, Wang, Jianmin, and Jordan, Michael I · 2016
Closest in time.
Wasserstein gan
Original
Arjovsky, Martin, Chintala, Soumith, and Bottou, Léon · 2017
Closest in time.
Adversarial discriminative domain adaptation
Original
Tzeng, Eric, Hoffman, Judy, Saenko, Kate, and Darrell, Trevor · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…