Fetching the paper…
Reading the bibliography…
Recent advancements in dataset distillation have demonstrated the significant benefits of employing soft labels generated by pre-trained teacher models.
A stochastic approximation method
Herbert Robbins and Sutton Monro · 1951
Earlier work this paper cites.
Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters
John Bridle · 1989
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Ya Le and Xuan Yang · 2015
Earlier work this paper cites.
Neural networks and deep learning , volume 25
Michael A Nielsen · 2015
Earlier work this paper cites.
Graph based constrained semi-supervised learning framework via label propagation over adaptive neighborhood
Zhao Zhang, Mingbo Zhao, and T. Chow · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna · 2016
Earlier work this paper cites.
Decoupled weight decay regularization
I Loshchilov · 2017
Earlier work this paper cites.
Representation learning with contrastive predictive coding
Aaron van den Oord, Yazhe Li, and Oriol Vinyals · 2018
Earlier work this paper cites.
Mobilenetv2: Inverted residuals and linear bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen · 2018
Earlier work this paper cites.
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros · 2018
Earlier work this paper cites.
Efficientnet: Rethinking model scaling for convolutional neural networks
Mingxing Tan and Quoc Le · 2019
Earlier work this paper cites.
Flexible dataset distillation: Learn labels instead of images
Ondrej Bohdal, Yongxin Yang, and Timothy Hospedales · 2020
Earlier work this paper cites.
Dataset meta-learning from kernel ridge-regression
Timothy Nguyen, Zhourong Chen, and Jaehoon Lee · 2020
Cited alongside, same era.
Gdumb: A simple approach that questions our progress in continual learning
Ameya Prabhu, Philip HS Torr, and Puneet K Dokania · 2020
Cited alongside, same era.
Learning soft labels via meta learning
Nidhi Vyas, S. Saxena, and T. Voice · 2020
Cited alongside, same era.
A theory of usable information under computational constraints
Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon · 2020
Cited alongside, same era.
Revisiting knowledge distillation via label smoothing regularization
Li Yuan, Francis EH Tay, Guilin Li, Tao Wang, and Jiashi Feng · 2020
Cited alongside, same era.
Synthesizing informative training samples with gan
Bo Zhao and Hakan Bilen · 2022
Later among the works it cites.
Decoupled knowledge distillation
Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang · 2022
Later among the works it cites.
Dataset distillation using neural feature regression
Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba · 2022
Later among the works it cites.
Generalizing dataset distillation via deep generative prior
George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu · 2023
Later among the works it cites.
Scaling up dataset distillation to imagenet-1k with constant memory
Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh · 2023
Later among the works it cites.
From knowledge distillation to self-knowledge distillation: A unified approach with normalized loss and customized soft labels
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Dataset condensation with gradient matching
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen · 2020
Cited alongside, same era.
Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation
Taehyeon Kim, Jaehoon Oh, NakYil Kim, Sangwook Cho, and Se-Young Yun · 2021
Cited alongside, same era.
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo · 2021
Cited alongside, same era.
Does knowledge distillation really work?
Samuel Stanton, Pavel Izmailov, Polina Kirichenko, Alexander A Alemi, and Andrew G Wilson · 2021
Cited alongside, same era.
Soft-label dataset distillation and text dataset distillation
Ilia Sucholutsky and Matthias Schonlau · 2021
Cited alongside, same era.
Dataset condensation with differentiable siamese augmentation
Bo Zhao and Hakan Bilen · 2021
Cited alongside, same era.
Rethinking soft labels for knowledge distillation: A bias-variance tradeoff perspective
Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou, Guoli Wang, Junsong Yuan, and Qian Zhang · 2021
Cited alongside, same era.
Zhendong Yang, Ailing Zeng, Zhe Li, Tianke Zhang, Chun Yuan, and Yu Li · 2023
Later among the works it cites.
Dataset distillation in large data era
Zeyuan Yin and Zhiqiang Shen · 2023
Later among the works it cites.
Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective
Zeyuan Yin, Eric Xing, and Zhiqiang Shen · 2023
Later among the works it cites.
White-box transformers via sparse rate reduction
Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Benjamin Haeffele, and Yi Ma · 2023
Later among the works it cites.
Dataset condensation with distribution matching
Bo Zhao and Hakan Bilen · 2023
Later among the works it cites.
Improved distribution matching for dataset condensation
Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu · 2023
Later among the works it cites.
Learning neural networks with sparse activations
Pranjal Awasthi, Nishanth Dikkala, Pritish Kamath, and Raghu Meka · 2024
Closest in time.
Towards lossless dataset distillation via difficulty-aligned trajectory matching
Ziyao Guo, Kai Wang, George Cazenavette, Hui Li, Kaipeng Zhang, and Yang You · 2024
Closest in time.
A label is worth a thousand images in dataset distillation
Tian Qin, Zhiwei Deng, and David Alvarez-Melis · 2024
Closest in time.
Generalized large-scale data condensation via various backbone and statistical matching
Shitong Shao, Zeyuan Yin, Muxin Zhou, Xindong Zhang, and Zhiqiang Shen · 2024
Closest in time.
On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm
Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin · 2024
Closest in time.