Fetching the paper…
Reading the bibliography…
For visual recognition, knowledge distillation typically involves transferring knowledge from a large, well-trained teacher model to a smaller student model.
Lxmert: Learning cross-modality encoder representations from transformers
Tan, H.; and Bansal, M. 2019 · 1908
Earlier work this paper cites.
Decoupling representation and classifier for long-tailed recognition
Kang, B.; Xie, S.; Rohrbach, M.; Yan, Z.; Gordo, A.; Feng, J.; and Kalantidis, Y. 2019 · 1910
Earlier work this paper cites.
Zhang, J.; Liu, L.; Wang, P.; and Shen, C. 2019 · 1912
Earlier work this paper cites.
A comprehensive overhaul of feature distillation
Heo, B.; Kim, J.; Yun, S.; Park, H.; Kwak, N.; and Choi, J. Y. 2019 · 1930
Earlier work this paper cites.
Addressing the curse of imbalanced training sets: one-sided selection
Kubat, M.; Matwin, S.; et al. 1997 · 1997
Earlier work this paper cites.
SMOTE: synthetic minority over-sampling technique
Chawla, N. V.; Bowyer, K. W.; Hall, L. O.; and Kegelmeyer, W. P. 2002 · 2002
Earlier work this paper cites.
A simple framework for contrastive learning of visual representations
Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020b · 2002
Earlier work this paper cites.
Knowledge distillation via adaptive instance normalization
Yang, J.; Martinez, B.; Bulat, A.; and Tzimiropoulos, G. 2020 · 2003
Earlier work this paper cites.
Hero: Hierarchical encoder for video+ language omni-representation pre-training
Li, L.; Chen, Y.-C.; Cheng, Y.; Gan, Z.; Yu, L.; and Liu, J. 2020a · 2005
Earlier work this paper cites.
Long-tail learning via logit adjustment
Menon, A. K.; Jayasumana, S.; Rawat, A. S.; Jain, H.; Veit, A.; and Kumar, S. 2020 · 2007
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2020 · 2010
Earlier work this paper cites.
Long-tailed recognition by routing diverse distribution-aware experts
Wang, X.; Lian, L.; Miao, Z.; Liu, Z.; and Yu, S. X. 2020b · 2010
Earlier work this paper cites.
Class imbalance, redux
Wallace, B. C.; Small, K.; Brodley, C. E.; and Trikalinos, T. A. 2011 · 2011
Earlier work this paper cites.
A closer look at the robustness of vision-and-language pre-trained models
Li, L.; Gan, Z.; and Liu, J. 2020 · 2012
Earlier work this paper cites.
On the statistical consistency of algorithms for binary classification under class imbalance
Menon, A.; Narasimhan, H.; Agarwal, S.; and Chawla, S. 2013 · 2013
Earlier work this paper cites.
Distilling the knowledge in a neural network
Hinton, G.; Vinyals, O.; and Dean, J. 2015 · 2015
Earlier work this paper cites.
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; Berg, A. C.; and Fei-Fei, L. 2015 · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016 · 2016
Earlier work this paper cites.
Learning deep representation for imbalanced classification
Huang, C.; Li, Y.; Loy, C. C.; and Tang, X. 2016 · 2016
Earlier work this paper cites.
Like what you like: Knowledge distill via neuron selectivity transfer
Huang, Z.; and Wang, N. 2017 · 2017
Earlier work this paper cites.
A gift from knowledge distillation: Fast optimization, network minimization and transfer learning
Yim, J.; Joo, D.; Bae, J.; and Kim, J. 2017 · 2017
Earlier work this paper cites.
Places: A 10 million image database for scene recognition
Zhou, B.; Lapedriza, A.; Khosla, A.; Oliva, A.; and Torralba, A. 2017 · 2017
Earlier work this paper cites.
Darkrank: Accelerating deep metric learning via cross sample similarities transfer
Chen, Y.; Wang, N.; and Zhang, Z. 2018 · 2018
Earlier work this paper cites.
Paraphrasing complex network: Network compression via factor transfer
Kim, J.; Park, S.; and Kwak, N. 2018 · 2018
Earlier work this paper cites.
The inaturalist species classification and detection dataset
Van Horn, G.; Mac Aodha, O.; Song, Y.; Cui, Y.; Sun, C.; Shepard, A.; Adam, H.; Perona, P.; and Belongie, S. 2018 · 2018
Earlier work this paper cites.
Unsupervised feature learning via non-parametric instance discrimination
Wu, Z.; Xiong, Y.; Yu, S. X.; and Lin, D. 2018 · 2018
Earlier work this paper cites.
Zhang, C.; and Peng, Y. 2018 · 2018
Cited alongside, same era.
Learning imbalanced datasets with label-distribution-aware margin loss
Cao, K.; Wei, C.; Gaidon, A.; Arechiga, N.; and Ma, T. 2019 · 2019
Cited alongside, same era.
Uniter: Learning universal image-text representations
Chen, Y.-C.; Li, L.; Yu, L.; El Kholy, A.; Ahmed, F.; Gan, Z.; Cheng, Y.; and Liu, J. 2019 · 2019
Cited alongside, same era.
Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks
Lu, J.; Batra, D.; Parikh, D.; and Lee, S. 2019 · 2019
Cited alongside, same era.
Amalgamating knowledge towards comprehensive classification
Shen, C.; Wang, X.; Song, J.; Sun, L.; and Song, M. 2019 · 2019
Cited alongside, same era.
Metasaug: Meta semantic augmentation for long-tailed visual recognition
Li, S.; Gong, K.; Liu, C. H.; Wang, Y.; Qiao, F.; and Cheng, X. 2021 · 2021
Later among the works it cites.
Micro-expression action unit detection with dual-view attentive similarity-preserving knowledge distillation
Li, Y.; Peng, W.; and Zhao, G. 2021 · 2021
Later among the works it cites.
A simple long-tailed recognition baseline via vision-language model
Ma, T.; Geng, S.; Wang, M.; Shao, J.; Lu, J.; Li, H.; Gao, P.; and Qiao, Y. 2021 · 2021
Later among the works it cites.
Alp-kd: Attention-based layer projection for knowledge distillation
Passban, P.; Wu, Y.; Rezagholizadeh, M.; and Liu, Q. 2021 · 2021
Later among the works it cites.
Learning transferable visual models from natural language supervision
Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021 · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Feature space augmentation for long-tailed data
Chu, P.; Bian, X.; Liu, S.; and Ling, H. 2020 · 2020
Cited alongside, same era.
Large-scale adversarial training for vision-and-language representation learning
Gan, Z.; Chen, Y.-C.; Li, L.; Zhu, C.; Cheng, Y.; and Liu, J. 2020 · 2020
Cited alongside, same era.
Bootstrap your own latent-a new approach to self-supervised learning
Grill, J.-B.; Strub, F.; Altché, F.; Tallec, C.; Richemond, P.; Buchatskaya, E.; Doersch, C.; Avila Pires, B.; Guo, Z.; Gheshlaghi Azar, M.; et al. 2020 · 2020
Cited alongside, same era.
Differentiable feature aggregation search for knowledge distillation
Guan, Y.; Zhao, P.; Wang, B.; Zhang, Y.; Yao, C.; Bian, K.; and Tang, J. 2020 · 2020
Cited alongside, same era.
Momentum contrast for unsupervised visual representation learning
He, K.; Fan, H.; Wu, Y.; Xie, S.; and Girshick, R. 2020 · 2020
Cited alongside, same era.
Exploring balanced feature spaces for representation learning
Kang, B.; Li, Y.; Xie, S.; Yuan, Z.; and Feng, J. 2020 · 2020
Cited alongside, same era.
Supervised contrastive learning
Khosla, P.; Teterwak, P.; Wang, C.; Sarna, A.; Tian, Y.; Isola, P.; Maschinot, A.; Liu, C.; and Krishnan, D. 2020 · 2020
Cited alongside, same era.
Distributional robustness loss for long-tail learning
Samuel, D.; and Chechik, G. 2021 · 2021
Later among the works it cites.
Fasa: Feature augmentation and sampling adaptation for long-tailed instance segmentation
Zang, Y.; Huang, C.; and Loy, C. C. 2021 · 2021
Later among the works it cites.
Improving calibration for long-tailed recognition
Zhong, Z.; Cui, J.; Liu, S.; and Jia, J. 2021 · 2021
Later among the works it cites.
Reslt: Residual learning for long-tailed recognition
Cui, J.; Liu, S.; Tian, Z.; Zhong, Z.; and Jia, J. 2022 · 2022
Later among the works it cites.
Glance and focus networks for dynamic visual recognition
Huang, G.; Wang, Y.; Lv, K.; Jiang, H.; Huang, W.; Qi, P.; and Song, S. 2022 · 2022
Later among the works it cites.
Long-tailed visual recognition via gaussian clouded logit adjustment
Li, M.; Cheung, Y.-m.; and Lu, Y. 2022 · 2022
Later among the works it cites.
Vl-ltr: Learning class-wise visual-linguistic representation for long-tailed visual recognition
Tian, C.; Wang, W.; Zhu, X.; Dai, J.; and Qiao, Y. 2022 · 2022
Later among the works it cites.
Towards calibrated hyper-sphere representation via distribution overlap coefficient for long-tailed learning
Wang, H.; Fu, S.; He, X.; Fang, H.; Liu, Z.; and Hu, H. 2022 · 2022
Later among the works it cites.
Constructing balance from imbalance for long-tailed image recognition
Xu, Y.; Li, Y.-L.; Li, J.; and Lu, C. 2022 · 2022
Later among the works it cites.
Balanced contrastive learning for long-tailed visual recognition
Zhu, J.; Wang, Z.; Chen, J.; Chen, Y.-P. P.; and Jiang, Y.-G. 2022 · 2022
Later among the works it cites.
Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023 · 2023
Later among the works it cites.
Sharegpt4v: Improving large multi-modal models with better captions
Chen, L.; Li, J.; Dong, X.; Zhang, P.; He, C.; Wang, J.; Zhao, F.; and Lin, D. 2023 · 2023
Later among the works it cites.
Generalized parametric contrastive learning
Cui, J.; Zhong, Z.; Tian, Z.; Liu, S.; Yu, B.; and Jia, J. 2023 · 2023
Later among the works it cites.
A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance
Huang, Z.; Zhou, A.; Ling, Z.; Cai, M.; Wang, H.; and Lee, Y. J. 2023 · 2023
Later among the works it cites.
Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; Casas, D. d. l.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; et al. 2023 · 2023
Later among the works it cites.
Learning Imbalanced Data with Vision Transformers
Xu, Z.; Liu, R.; Yang, S.; Chai, Z.; and Yuan, C. 2023 · 2023
Later among the works it cites.
Probabilistic Contrastive Learning for Long-Tailed Visual Recognition
Du, C.; Wang, Y.; Song, S.; and Huang, G. 2024 · 2024
Closest in time.
LLaVA-NeXT: Improved reasoning, OCR, and world knowledge
Liu, H.; Li, C.; Li, Y.; Li, B.; Zhang, Y.; Shen, S.; and Lee, Y. J. 2024 · 2024
Closest in time.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Reid, M.; Savinov, N.; Teplyashin, D.; Lepikhin, D.; Lillicrap, T.; Alayrac, J.-b.; Soricut, R.; Lazaridou, A.; Firat, O.; Schrittwieser, J.; et al. 2024 · 2024
Closest in time.
Probabilistic knowledge transfer for lightweight deep representation learning
Passalis, N.; Tzelepi, M.; and Tefas, A. 2020b · 2039
Closest in time.