2021

Few-Shot Learning with Class Imbalance

Ochal, Mateusz, Patacchiola, Massimiliano, Storkey, Amos et al.

Understand

Few-Shot Learning (FSL) algorithms are commonly trained through Meta-Learning (ML), which exposes models to batches of tasks sampled from a meta-dataset to mimic tasks seen during evaluation.

However, the standard training procedures overlook the real-world dynamics where classes commonly occur at different frequencies.
While it is generally understood that class imbalance harms the performance of supervised methods, limited research examines the impact of imbalance on the FSL evaluation task.
Our analysis compares 10 state-of-the-art meta-learning and FSL methods on different imbalance distributions and rebalancing techniques.

Built on

W. J. Reed, “The pareto, zipf and other power laws,” Economics Letters , 2001
2001
Earlier work this paper cites.
N. Japkowicz and S. Stephen, “The class imbalance problem: A systematic study,” Intelligent Data Analysis , vol. 6, pp. 429–449, 2002
2002
Earlier work this paper cites.
S. J. Pan and Q. Yang, “A Survey on Transfer Learning,” IEEE Transactions on Knowledge and Data Engineering , vol. 22, pp. 1345–1359, 2010
2010
Earlier work this paper cites.
R. Salakhutdinov, A. Torralba, and J. Tenenbaum, “Learning to share visual appearance for multiclass object detection,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2011
2011
Earlier work this paper cites.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The Caltech-UCSD Birds-200-2011 Dataset,” in California Institute of Technology , 2011
2011
Earlier work this paper cites.
S. Wang, L. L. Minku, and X. Yao, “Resampling-based ensemble methods for online class imbalance learning,” IEEE Transactions on Knowledge and Data Engineering , vol. 27, no. 5, pp. 1356–1368, 2014
2014
Earlier work this paper cites.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521, pp. 436–444, 2015
2015
Earlier work this paper cites.
S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” International Conference on Machine Learning (ICML) , 2015
2015
Earlier work this paper cites.
S. Ravi and H. Larochelle, “Optimization as a model for few-shot learning,” International Conference on Learning Representations (ICLR) , 2016
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016
2016
Earlier work this paper cites.
O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra, “Matching Networks for One Shot Learning,” Advances in Neural Information Processing Systems (NeurIPS) , 2017
2017
Earlier work this paper cites.
T.-y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal Loss for Dense Object Detection,” IEEE International Conference on Computer Vision (ICCV) , 2017
2017
Earlier work this paper cites.
M. Havaei, A. Davy, D. Warde-Farley, A. Biard, A. Courville, Y. Bengio, C. Pal, P. M. Jodoin, and H. Larochelle, “Brain tumor segmentation with Deep Neural Networks,” Medical Image Analysis , vol. 35, pp. 18–31, 2017
2017
Earlier work this paper cites.
J. Snell, K. Swersky, and R. S. Zemel, “Prototypical Networks for Few-shot Learning,” Advances in Neural Information Processing Systems (NeurIPS) , 2017
2017
Earlier work this paper cites.
F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. S. Torr, and T. M. Hospedales, “Learning to Compare: Relation Network for Few-Shot Learning,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017
2017
Earlier work this paper cites.
H. Edwards and A. Storkey, “Towards a Neural Statistician,” International Conference on Learning Representations (ICLR) , 2017
2017
Earlier work this paper cites.
C. Finn, P. Abbeel, and S. Levine, “Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks,” International Conference on Machine Learning (ICML) , 2017
2017
Earlier work this paper cites.
A. Antoniou, A. Storkey, and H. Edwards, “Data Augmentation Generative Adversarial Networks,” SSRN Electronic Journal , 2017
2017
Earlier work this paper cites.
B. Hariharan and R. Girshick, “Low-Shot Visual Recognition by Shrinking and Hallucinating Features,” IEEE International Conference on Computer Vision (ICCV) , 2017
2017
Earlier work this paper cites.
Z. Li, F. Zhou, F. Chen, and H. Li, “Meta-SGD: Learning to Learn Quickly for Few-Shot Learning,” arXiv preprint arXiv:1707.09835 , 2017
Original
2017
Earlier work this paper cites.

Similar

M. Buda, A. Maki, and M. A. Mazurowski, “A systematic study of the class imbalance problem in convolutional neural networks,” Neural Networks , vol. 106, pp. 249–259, 2018
2018
Cited alongside, same era.
J. L. Leevy, T. M. Khoshgoftaar, R. A. Bauder, and N. Seliya, “A survey on addressing high-class imbalance in big data,” Journal of Big Data , vol. 5, 2018
2018
Cited alongside, same era.
F. Chen, M. Luo, Z. Dong, Z. Li, and X. He, “Federated meta-learning with fast convergence and efficient communication,” arXiv preprint arXiv:1802.07876 , 2018
Original
2018
Cited alongside, same era.
R. Zhang, T. Che, Z. Ghahramani, Y. Song, Y. Bengio, and Y. Song, “MetaGAN : An Adversarial Approach to Few-Shot Learning,” Advances in Neural Information Processing Systems (NeurIPS) , 2018
2018
Cited alongside, same era.
M. Ren, R. Liao, E. Fetaya, and R. S. Zemel, “Incremental Few-Shot Learning with Attention Attractor Networks,” arXiv preprint arXiv:1810.07218 , 2018
Original
2018
Cited alongside, same era.
S. Gidaris and N. Komodakis, “Dynamic Few-Shot Visual Learning Without Forgetting,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018
2018
Cited alongside, same era.
A. Antoniou, H. Edwards, and A. Storkey, “How to train your maml,” arXiv preprint arXiv:1810.09502 , 2018
Original
2018
Cited alongside, same era.
Z.-Q. Cheng, X. Wu, S. Huang, J.-X. Li, A. G. Hauptmann, and Q. Peng, “Learning to Transfer Learn,” in 2018 ACM Multimedia Conference on Multimedia Conference - MM ’18 . ACM Press, 2018, pp. 90–98
2018
Cited alongside, same era.
J. Yoon, T. Kim, O. Dia, S. Kim, Y. Bengio, and S. Ahn, “Bayesian model-agnostic meta-learning,” Advances in Neural Information Processing Systems (NeurIPS) , 2018
2018
Cited alongside, same era.
Z. Liu, Z. Miao, X. Zhan, J. Wang, B. Gong, and S. X. Yu, “Large-Scale Long-Tailed Recognition in an Open World,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 2537–2546, 2019
2019
Cited alongside, same era.
H. B. Lee, H. Lee, D. Na, S. Kim, M. Park, E. Yang, and S. J. Hwang, “Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks,” International Conference on Machine Learning (ICML) , 2019
2019
Cited alongside, same era.
D. Wertheimer and B. Hariharan, “Few-Shot Learning with Localization in Realistic Settings,” arXiv preprint arXiv:1904.08502 , pp. 6558–6567, 2019
Original
2019
Cited alongside, same era.

Then

A. Li, T. Luo, T. Xiang, W. Huang, and L. Wang, “Few-shot learning with global class representations,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2019
2019
Later among the works it cites.
Y. Wang, W.-L. Chao, K. Q. Weinberger, and L. van der Maaten, “SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning,” arXiv preprint arXiv:1911.04623 , 2019
Original
2019
Later among the works it cites.
A. Antoniou and A. Storkey, “Assume, Augment and Learn: Unsupervised Few-Shot Meta-Learning via Random Labels and Data Augmentation,” arXiv preprint arXiv:1902.09884 , 2019
Original
2019
Later among the works it cites.
M. Ochal, J. Vazquez, Y. Petillot, and S. Wang, “A comparison of few-shot learning methods for underwater optical and sonar image classification,” in Global Oceans 2020: Singapore – U.S. Gulf Coast , 2020, pp. 1–10
2020
Later among the works it cites.
J. Guan, J. Liu, J. Sun, P. Feng, T. Shuai, and W. Wang, “Meta Metric Learning for Highly Imbalanced Aerial Scene Classification,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2020, pp. 4047–4051
2020
Later among the works it cites.
L. Zhang, C. Zhang, S. Quan, H. Xiao, G. Kuang, L. Liu, and L. Liu, “A class imbalance loss for imbalanced object recognition,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 13, pp. 2778–2792, 2020
2020
Later among the works it cites.
E. Triantafillou, T. Zhu, V. Dumoulin, P. Lamblin, U. Evci, K. Xu, R. Goroshin, C. Gelada, K. Swersky, P.-A. Manzagol, and H. Larochelle, “Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples,” International Conference on Learning Representations (ICLR) , 2020
2020
Later among the works it cites.
X. Chen, H. Dai, Y. Li, X. Gao, and L. Song, “Learning to Stop While Learning to Predict,” International Conference on Machine Learning (ICML) , 2020
2020
Later among the works it cites.
G. S. Dhillon, P. Chaudhari, A. Ravichandran, and S. Soatto, “A Baseline for Few-Shot Image Classification,” in International Conference on Learning Representations , 2020
2020
Later among the works it cites.
M. Patacchiola, J. Turner, E. J. Crowley, M. O’Boyle, and A. Storkey, “Bayesian Meta-Learning in the Few-Shot Setting via Deep Kernels,” in Advances in Neural Information Processing Systems (NeurIPS) , 2020
2020
Later among the works it cites.
M. Boudiaf, Z. I. Masud, J. Rony, J. Dolz, P. Piantanida, and I. B. Ayed, “Transductive information maximization for few-shot learning,” 2020
2020
Later among the works it cites.
T. Cao, M. T. Law, and S. Fidler, “A theoretical analysis of the number of shots in few-shot learning,” in International Conference on Learning Representations (ICLR) , 2020
2020
Later among the works it cites.
J. Rajendran, A. Irpan, and E. Jang, “Meta-learning requires meta-augmentation,” Advances in Neural Information Processing Systems (NeurIPS) , 2020
2020
Later among the works it cites.
R. Ni, M. Goldblum, A. Sharaf, K. Kong, and T. Goldstein, “Data augmentation for meta-learning,” Advances in Neural Information Processing Systems (NeurIPS) - Workshop on Meta-Learning , 2020
2020
Later among the works it cites.
A. Antoniou, M. Patacchiola, M. Ochal, and A. Storkey, “Defining Benchmarks for Continual Few-Shot Learning,” arXiv preprint arXiv:2004.11967 , 2020
Original
2020
Later among the works it cites.
J. Snell and R. Zemel, “Bayesian Few-Shot Classification with One-vs-Each Polya-Gamma Augmented Gaussian Processes,” arXiv preprint arXiv:2007.10417 , 2020
Original
2020
Later among the works it cites.
E. Vogelbaum, R. Dangovski, L. Jing, and M. Soljačić, “Contextualizing Enhances Gradient Based Meta Learning,” arXiv preprint arXiv:2007.10143 , 2020
Original
2020
Later among the works it cites.
D. Massiceti, L. Zintgraf, J. Bronskill, L. Theodorou, M. T. Harris, E. Cutrell, C. Morrison, K. Hofmann, and S. Stumpf, “Orbit: A real-world few-shot dataset for teachable object recognition,” arXiv preprint arXiv:2104.03841 , 2021
Original
2021
Closest in time.
X. Zhang, D. Meng, H. Gouk, and T. Hospedales, “Shallow bayesian meta learning for real-world few-shot recognition,” arXiv preprint arXiv:2101.02833 , 2021
Original
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…