2019

Generalizing to unseen domains via distribution matching

Albuquerque, Isabela, Monteiro, João, Darvishi, Mohammad et al.

Understand

Supervised learning results typically rely on assumptions of i.i.d.

data.
Unfortunately, those assumptions are commonly violated in practice.
In this work, we tackle such problem by focusing on domain generalization: a formalization where the data generating process at test time may yield samples from never-before-seen domains (distributions).

Built on

D. Kifer, S. Ben-David, and J. Gehrke, “Detecting change in data streams,” in Proceedings of the Thirtieth international conference on Very large data bases-Volume 30 . VLDB Endowment, 2004, pp. 180–191
2004
Earlier work this paper cites.
J. Langford, “Tutorial on practical prediction theory for classification,” Journal of machine learning research , vol. 6, no. Mar, pp. 273–306, 2005
2005
Earlier work this paper cites.
S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira, “Analysis of representations for domain adaptation,” in Advances in neural information processing systems , 2007, pp. 137–144
2007
Earlier work this paper cites.
G. Griffin, A. Holub, and P. Perona, “Caltech-256 object category dataset,” 2007
2007
Earlier work this paper cites.
B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “Labelme: a database and web-based tool for image annotation,” International journal of computer vision , vol. 77, no. 1-3, pp. 157–173, 2008
2008
Earlier work this paper cites.
M. Dredze, A. Kulesza, and K. Crammer, “Multi-domain learning by confidence-weighted parameter combination,” Machine Learning , vol. 79, no. 1-2, pp. 123–149, 2010
2010
Earlier work this paper cites.
S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan, “A theory of learning from different domains,” Machine learning , vol. 79, no. 1-2, pp. 151–175, 2010
2010
Earlier work this paper cites.
S. Ben-David, T. Lu, T. Luu, and D. Pál, “Impossibility theorems for domain adaptation,” in International Conference on Artificial Intelligence and Statistics , 2010, pp. 129–136
2010
Earlier work this paper cites.
M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision , vol. 88, no. 2, pp. 303–338, 2010
2010
Earlier work this paper cites.
M. J. Choi, J. J. Lim, A. Torralba, and A. S. Willsky, “Exploiting hierarchical context on a large database of object categories,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . IEEE, 2010, pp. 129–136
2010
Earlier work this paper cites.
A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola, “A kernel two-sample test,” The Journal of Machine Learning Research , vol. 13, no. 1, pp. 723–773, 2012
2012
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems , 2012, pp. 1097–1105
2012
Earlier work this paper cites.
K. Muandet, D. Balduzzi, and B. Schölkopf, “Domain generalization via invariant feature representation,” in International Conference on Machine Learning , 2013, pp. 10–18
2013
Earlier work this paper cites.
C. Fang, Y. Xu, and D. N. Rockmore, “Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias,” in Proceedings of the IEEE International Conference on Computer Vision , 2013, pp. 1657–1664
2013
Earlier work this paper cites.
Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in International conference on machine learning . PMLR, 2015, pp. 1180–1189
2015
Earlier work this paper cites.
S. Sun, H. Shi, and Y. Wu, “A survey of multi-source domain adaptation,” Information Fusion , vol. 24, pp. 84–92, 2015
2015
Earlier work this paper cites.

Similar

D. Wu, V. J. Lawhern, and B. J. Lance, “Reducing bci calibration effort in rsvp tasks using online weighted adaptation regularization with source domain selection,” in 2015 International Conference on Affective Computing and Intelligent Interaction (ACII) . IEEE, 2015, pp. 567–573
2015
Cited alongside, same era.
W.-L. Zheng and B.-L. Lu, “Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks,” IEEE Transactions on Autonomous Mental Development , vol. 7, no. 3, pp. 162–175, 2015
2015
Cited alongside, same era.
M. Long, Y. Cao, J. Wang, and M. Jordan, “Learning transferable features with deep adaptation networks,” in International Conference on Machine Learning , 2015, pp. 97–105
2015
Cited alongside, same era.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV) , vol. 115, no. 3, pp. 211–252, 2015
2015
Cited alongside, same era.
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” The Journal of Machine Learning Research , vol. 17, no. 1, pp. 2096–2030, 2016
2016
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Cited alongside, same era.
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 2818–2826
2016
Cited alongside, same era.
D. Li, Y. Yang, Y.-Z. Song, and T. M. Hospedales, “Deeper, broader and artier domain generalization,” in Proceedings of the IEEE International Conference on Computer Vision , 2017, pp. 5542–5550
2017
Cited alongside, same era.
B. Neyshabur, S. Bhojanapalli, and A. Chakrabarti, “Stabilizing gan training with multiple random projections,” arXiv preprint arXiv:1705.07831 , 2017
Original
2017
Cited alongside, same era.
Y. Li, K. Dzirasa, L. Carin, D. E. Carlson et al. , “Targeting eeg/lfp synchrony with neural nets,” in Advances in Neural Information Processing Systems , 2017, pp. 4620–4630
2017
Cited alongside, same era.
S. Shankar, V. Piratla, S. Chakrabarti, S. Chaudhuri, P. Jyothi, and S. Sarawagi, “Generalizing across domains via cross-gradient training,” in International Conference on Learning Representations , 2018. [Online]. Available: https://openreview.net/forum?id=r1Dx7fbCW
2018
Cited alongside, same era.
R. Volpi, H. Namkoong, O. Sener, J. C. Duchi, V. Murino, and S. Savarese, “Generalizing to unseen domains via adversarial data augmentation,” in Advances in Neural Information Processing Systems , 2018, pp. 5334–5344
2018
Cited alongside, same era.

Then

H. Zhao, S. Zhang, G. Wu, J. M. Moura, J. P. Costeira, and G. J. Gordon, “Adversarial multiple source domain adaptation,” in Advances in Neural Information Processing Systems , 2018, pp. 8559–8570
2018
Later among the works it cites.
J. Hoffman, M. Mohri, and N. Zhang, “Algorithms and theory for multiple-source adaptation,” in Advances in Neural Information Processing Systems , 2018, pp. 8246–8256
2018
Later among the works it cites.
C.-S. Wei, Y.-P. Lin, Y.-T. Wang, C.-T. Lin, and T.-P. Jung, “A subject-transfer framework for obviating inter-and intra-subject variability in eeg-based drowsiness detection,” NeuroImage , vol. 174, pp. 407–419, 2018
2018
Later among the works it cites.
Y. Li, D. E. Carlson et al. , “Extracting relationships by multi-domain matching,” in Advances in Neural Information Processing Systems , 2018, pp. 6798–6809
2018
Later among the works it cites.
F. M. Carlucci, A. D’Innocente, S. Bucci, B. Caputo, and T. Tommasi, “Domain generalization by solving jigsaw puzzles,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 2229–2238
2019
Closest in time.
M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz, “Invariant risk minimization,” arXiv preprint arXiv:1907.02893 , 2019
Original
2019
Closest in time.
Q. Dou, D. C. de Castro, K. Kamnitsas, and B. Glocker, “Domain generalization via model-agnostic learning of semantic features,” in Advances in Neural Information Processing Systems , 2019, pp. 6447–6458
2019
Closest in time.
D. Li, J. Zhang, Y. Yang, C. Liu, Y.-Z. Song, and T. M. Hospedales, “Episodic training for domain generalization,” arXiv preprint arXiv:1902.00113 , 2019
Original
2019
Closest in time.
A. Schoenauer-Sebag, L. Heinrich, M. Schoenauer, M. Sebag, L. F. Wu, and S. J. Altschuler, “Multi-domain adversarial learning,” arXiv preprint arXiv:1903.09239 , 2019
Original
2019
Closest in time.
H. Zhao, R. T. Des Combes, K. Zhang, and G. Gordon, “On learning invariant representations for domain adaptation,” in International Conference on Machine Learning , 2019, pp. 7523–7532
2019
Closest in time.
I. Albuquerque, J. Monteiro, T. Doan, B. Considine, T. Falk, and I. Mitliagkas, “Multi-objective training of generative adversarial networks with multiple discriminators,” in International Conference on Machine Learning , 2019, pp. 202–211
2019
Closest in time.
X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang, “Moment matching for multi-source domain adaptation,” in Proceedings of the IEEE International Conference on Computer Vision , 2019, pp. 1406–1415
2019
Closest in time.
I. Albuquerque, J. Monteiro, O. Rosanne, A. Tiwari, J.-F. Gagnon, and T. H. Falk, “Cross-subject statistical shift estimation for generalized electroencephalography-based mental workload assessment,” in 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) . IEEE, 2019, pp. 3647–3653
2019
Closest in time.
I. Albuquerque, N. Naik, J. Li, N. Keskar, and R. Socher, “Improving out-of-distribution generalization via multi-task self-supervised pretraining,” arXiv preprint arXiv:2003.13525 , 2020
Original
2020
Closest in time.
D. Mahajan, S. Tople, and A. Sharma, “Domain generalization using causal matching,” arXiv preprint arXiv:2006.07500 , 2020
Original
2020
Closest in time.
K. Ahuja, K. Shanmugam, K. Varshney, and A. Dhurandhar, “Invariant risk minimization games,” arXiv preprint arXiv:2002.04692 , 2020
Original
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…