Fetching the paper…
Reading the bibliography…
The lightweight "local-match-global" matching introduced by SRe2L successfully creates a distilled dataset with comprehensive information on the full 224x224 ImageNet-1k.
Visualizing data using t-SNE
Laurens Van der Maaten and Geoffrey Hinton · 2008
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
Distilling the knowledge in a neural network, 2015
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, and Shaoqing Ren · 2016
Earlier work this paper cites.
Wide residual networks
Sergey Zagoruyko and Nikos Komodakis · 2016
Earlier work this paper cites.
Learning from multiple teacher networks
Shan You, Chang Xu, Chao Xu, and Dacheng Tao · 2017
Earlier work this paper cites.
Mobilenetv2: Inverted residuals and linear bottlenecks
Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen · 2018
Earlier work this paper cites.
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros · 2018
Earlier work this paper cites.
Shufflenet: An extremely efficient convolutional neural network for mobile devices
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun · 2018
Earlier work this paper cites.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al · 2019
Earlier work this paper cites.
Contrastive representation distillation
Yonglong Tian, Dilip Krishnan, and Phillip Isola · 2019
Earlier work this paper cites.
Reducing catastrophic forgetting with learning on synthetic data
Wojciech Masarczyk and Ivona Tautkute · 2020
Earlier work this paper cites.
Dataset meta-learning from kernel ridge-regression
Timothy Nguyen, Zhourong Chen, and Jaehoon Lee · 2020
Earlier work this paper cites.
Generative teaching networks: Accelerating neural architecture search by learning to generate synthetic training data
Felipe Petroski Such, Aditya Rawal, Joel Lehman, Kenneth O. Stanley, and Jeffrey Clune · 2020
Earlier work this paper cites.
Embedded encoder-decoder in convolutional networks towards explainable AI
Amirhossein Tavanaei · 2020
Cited alongside, same era.
Dreaming to distill: Data-free knowledge transfer via deepinversion
Hongxu Yin, Pavlo Molchanov, Jose M Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K Jha, and Jan Kautz · 2020
Cited alongside, same era.
Knowledge distillation: A survey
Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao · 2021
Cited alongside, same era.
Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation
Taehyeon Kim, Jaehoon Oh, NakYil Kim, Sangwook Cho, and Se-Young Yun · 2021
Cited alongside, same era.
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo · 2021
Cited alongside, same era.
Efficientnetv2: Smaller models and faster training
A fast knowledge distillation framework for visual recognition
Zhiqiang Shen and Eric Xing · 2022
Later among the works it cites.
Cafe: Learning to condense dataset by aligning features
Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You · 2022
Later among the works it cites.
Decoupled knowledge distillation
Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang · 2022
Later among the works it cites.
Dataset distillation using neural feature regression
Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba · 2022
Later among the works it cites.
Scaling up dataset distillation to imagenet-1k with constant memory
Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh · 2023
Closest in time.
Minimizing the accumulated trajectory error to improve dataset distillation
Jiawei Du, Yidi Jiang, Vincent Y. F. Tan, Joey Tianyi Zhou, and Haizhou Li · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Mingxing Tan and Quoc Le · 2021
Cited alongside, same era.
Training data-efficient image transformers & distillation through attention
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou · 2021
Cited alongside, same era.
Pointaugmenting: Cross-modal augmentation for 3d object detection
Chunwei Wang, Chao Ma, Ming Zhu, and Xiaokang Yang · 2021
Cited alongside, same era.
Dataset condensation with differentiable siamese augmentation
Bo Zhao and Hakan Bilen · 2021
Cited alongside, same era.
Dataset condensation with gradient matching
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen · 2021
Cited alongside, same era.
Dataset distillation by matching training trajectories
George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A. Efros, and Jun-Yan Zhu · 2022
Cited alongside, same era.
Fedsynth: Gradient compression via synthetic data in federated learning
Shengyuan Hu, Jack Goetz, Kshitiz Malik, Hongyuan Zhan, Zhe Liu, and Yue Liu · 2022
Cited alongside, same era.
Closest in time.
DREAM: efficient dataset distillation by representative matching
Yanqing Liu, Jianyang Gu, Kai Wang, Zheng Zhu, Wei Jiang, and Yang You · 2023
Closest in time.
Dreamfusion: Text-to-3d using 2d diffusion
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall · 2023
Closest in time.
Noveen Sachdeva and Julian McAuley · 2023
Closest in time.
Datadam: Efficient dataset distillation with attention matching
Ahmad Sajedi, Samir Khaki, Ehsan Amjadian, Lucy Z Liu, Yuri A Lawryshyn, and Konstantinos N Plataniotis · 2023
Closest in time.
Squeeze, recover and relabel: Dataset condensation at imagenet scale from A new perspective
Zeyuan Yin, Eric P. Xing, and Zhiqiang Shen · 2023
Closest in time.
Dataset distillation: A comprehensive review
Ruonan Yu, Songhua Liu, and Xinchao Wang · 2023
Closest in time.
Accelerating dataset distillation via model augmentation
Lei Zhang, Jie Zhang, Bowen Lei, Subhabrata Mukherjee, Xiang Pan, Bo Zhao, Caiwen Ding, Yao Li, and Dongkuan Xu · 2023
Closest in time.
Dataset condensation with distribution matching
Bo Zhao and Hakan Bilen · 2023
Closest in time.