2021

Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

Zhang, Xueting, Meng, Debin, Gouk, Henry et al.

Understand

Current state-of-the-art few-shot learners focus on developing effective training procedures for feature representations, before using simple, e.g.

nearest centroid, classifiers.
In this paper, we take an orthogonal approach that is agnostic to the features used and focus exclusively on meta-learning the actual classifier layer.
Specifically, we introduce MetaQDA, a Bayesian meta-learning generalization of the classic quadratic discriminant analysis.

Built on

The evidence framework applied to classification networks
D. J. C. MacKay · 1992
Earlier work this paper cites.
A model of inductive bias learning
Jonathan Baxter · 2000
Earlier work this paper cites.
Empirical bayes for learning to learn
Tom Heskes · 2000
Earlier work this paper cites.
The elements of statistical learning
Jerome Friedman, Trevor Hastie, and Robert Tibshirani · 2001
Earlier work this paper cites.
Bayesian Data Analysis
Andrew Gelman, John B. Carlin, Hal S. Stern, and Donald B. Rubin · 2003
Earlier work this paper cites.
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
Trevor Hastie, Robert Tibshirani, and Jerome Friedman · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
Mnist handwritten digit database
Y Lecun and C Cortes · 2010
Earlier work this paper cites.
Machine learning: a probabilistic perspective
Kevin P Murphy · 2012
Earlier work this paper cites.
3d object representations for fine-grained categorization
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei · 2013
Earlier work this paper cites.
How transferable are features in deep neural networks?
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson · 2014
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Matching networks for one shot learning
Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra, et al · 2016
Earlier work this paper cites.
Model-agnostic meta-learning for fast adaptation of deep networks
Chelsea Finn, Pieter Abbeel, and Sergey Levine · 2017
Earlier work this paper cites.
On calibration of modern neural networks
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger · 2017
Earlier work this paper cites.
Meta-sgd: Learning to learn quickly for few-shot learning
Original
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li · 2017
Earlier work this paper cites.
Optimization as a model for few-shot learning
Sachin Ravi and Hugo Larochelle · 2017
Earlier work this paper cites.
iCaRL: Incremental classifier and representation learning
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert · 2017
Earlier work this paper cites.
A simple neural network module for relational reasoning
Adam Santoro, David Raposo, David GT Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap · 2017
Earlier work this paper cites.
Prototypical networks for few-shot learning
Jake Snell, Kevin Swersky, and Richard S Zemel · 2017
Earlier work this paper cites.
Few-shot learning with graph neural networks
Victor Garcia and Joan Bruna · 2018
Earlier work this paper cites.
Conditional neural processes
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and S. M. Ali Eslami · 2018
Earlier work this paper cites.

Similar

Dynamic few-shot visual learning without forgetting
Spyros Gidaris and Nikos Komodakis · 2018
Cited alongside, same era.
Recasting gradient-based meta-learning as hierarchical bayes
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Tom Griffiths · 2018
Cited alongside, same era.
Few-shot learning with metric-agnostic conditional embeddings
Original
Nathan Hilliard, Lawrence Phillips, Scott Howland, Artëm Yankov, Courtney D Corley, and Nathan O Hodas · 2018
Cited alongside, same era.
Toward Scalable Verification for Safety-Critical Deep Networks
Lindsey Kuper, Guy Katz, Justin Gottschlich, Kyle Julian, Clark Barrett, and Mykel Kochenderfer · 2018
Cited alongside, same era.
A simple neural attentive meta-learner
Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel · 2018
Cited alongside, same era.
Tadam: Task dependent adaptive metric for improved few-shot learning
Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste · 2018
Cited alongside, same era.
Few-shot image recognition by predicting parameters from activations
Siyuan Qiao, Chenxi Liu, Wei Shen, and Alan L Yuille · 2018
Cited alongside, same era.
Meta-learning for semi-supervised few-shot classification
Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel · 2018
Cited alongside, same era.
Learning to compare: Relation network for few-shot learning
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales · 2018
Cited alongside, same era.
Bayesian model-agnostic meta-learning
Jaesik Yoon, Taesup Kim, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn · 2018
Cited alongside, same era.
Meta-learning with differentiable closed-form solvers
Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi · 2019
Cited alongside, same era.
A closer look at few-shot classification
Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Wang, and Jia-Bin Huang · 2019
Cited alongside, same era.

Then

Meta-transfer learning for few-shot learning
Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele · 2019
Later among the works it cites.
Meta-dataset: A dataset of datasets for learning to learn from few examples
Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Evci, et al · 2019
Later among the works it cites.
Simpleshot: Revisiting nearest-neighbor classification for few-shot learning
Original
Yan Wang, Wei-Lun Chao, Kilian Q Weinberger, and Laurens van der Maaten · 2019
Later among the works it cites.
Adaptive cross-modal few-shot learning
Chen Xing, Negar Rostamzadeh, Boris Oreshkin, and Pedro O O Pinheiro · 2019
Later among the works it cites.
Tapnet: Neural network augmented with task-adaptive projection for few-shot learning
Sung Whan Yoon, Jun Seo, and Jaekyun Moon · 2019
Later among the works it cites.
Fast context adaptation via meta-learning
Luisa M Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson · 2019
Later among the works it cites.
Improved few-shot visual classification
Peyman Bateni, Raghav Goyal, Vaden Masrani, Frank Wood, and Leonid Sigal · 2020
Later among the works it cites.
Fairness in deep learning: A computational perspective
M. Du, F. Yang, N. Zou, and X. Hu · 2020
Later among the works it cites.
Selecting relevant features from a multi-domain representation for few-shot classification
Nikita Dvornik, Cordelia Schmid, and Julien Mairal · 2020
Later among the works it cites.
Unraveling meta-learning: Understanding feature representations for few-shot tasks
Micah Goldblum, Steven Reich, Liam Fowl, Renkun Ni, Valeriia Cherepanova, and Tom Goldstein · 2020
Later among the works it cites.
A broader study of cross-domain few-shot learning
Yunhui Guo, Noel C. F. Codella, Leonid Karlinsky, John R. Smith, Tajana Rosing, and Rogerio Feris · 2020
Later among the works it cites.
Meta-learning in neural networks: A survey
Original
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey · 2020
Later among the works it cites.
Adversarial feature hallucination networks for few-shot learning
Kai Li, Yulun Zhang, Kunpeng Li, and Yun Fu · 2020
Later among the works it cites.
Charting the right manifold: Manifold mixup for few-shot learning
Puneet Mangla, Nupur Kumari, Abhishek Sinha, Mayank Singh, Balaji Krishnamurthy, and Vineeth N Balasubramanian · 2020
Later among the works it cites.
Bayesian meta-learning for the few-shot setting via deep kernels
Massimiliano Patacchiola, Jack Turner, Elliot J Crowley, Michael O’Boyle, and Amos J Storkey · 2020
Later among the works it cites.
Explanation-guided training for cross-domain few-shot classification
Original
Jiamei Sun, Sebastian Lapuschkin, Wojciech Samek, Yunqing Zhao, Ngai-Man Cheung, and Alexander Binder · 2020
Later among the works it cites.
Few-shot class-incremental learning
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong · 2020
Later among the works it cites.
Cross-domain few-shot classification via learned feature-wise transformation
Hung-Yu Tseng, Hsin-Ying Lee, Jia-Bin Huang, and Ming-Hsuan Yang · 2020
Later among the works it cites.
A comparison of machine learning methods for cross-domain few-shot learning
Hongyu Wang, Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael Mayo · 2020
Later among the works it cites.
Generalizing from a few examples: A survey on few-shot learning
Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni · 2020
Later among the works it cites.
Meta-learning without memorization
Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, and Chelsea Finn · 2020
Later among the works it cites.
RelationNet2: deep comparison columns for few-shot learning
Xueting Zhang, Yuting Qiang, Sung Flood, Yongxin Yang, and Timothy M. Hospedales · 2020
Later among the works it cites.
A universal representation transformer layer for few-shot image classification
Lu Liu, William Hamilton, Guodong Long, Jing Jiang, and Hugo Larochelle · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…