Fetching the paper…
Reading the bibliography…
Data $\textit{quality}$ is a crucial factor in the performance of machine learning models, a principle that dataset distillation methods exploit by compressing training datasets into much smaller counterparts that maintain similar downstream performance.
“When does label smoothing help?”
Rafael Müller, Simon Kornblith and Geoffrey Hinton · 1906
Earlier work this paper cites.
“When does label smoothing help?”
Rafael Müller, Simon Kornblith and Geoffrey Hinton · 1906
Earlier work this paper cites.
“Contrastive Representation Distillation”
Yonglong Tian, Dilip Krishnan and Phillip Isola · 1910
Earlier work this paper cites.
“Contrastive Representation Distillation”
Yonglong Tian, Dilip Krishnan and Phillip Isola · 1910
Earlier work this paper cites.
“PyTorch: An Imperative Style, High-Performance Deep Learning Library”
Adam Paszke et al · 1912
Earlier work this paper cites.
“Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion”
Hongxu Yin et al · 1912
Earlier work this paper cites.
“PyTorch: An Imperative Style, High-Performance Deep Learning Library”
Adam Paszke et al · 1912
Earlier work this paper cites.
“Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion”
Hongxu Yin et al · 1912
Earlier work this paper cites.
“WordNet: a lexical database for English”
George Miller · 1995
Earlier work this paper cites.
“WordNet: a lexical database for English”
George Miller · 1995
Earlier work this paper cites.
“Flexible Dataset Distillation: Learn Labels Instead of Images”
Ondrej Bohdal, Yongxin Yang and Timothy Hospedales · 2006
Earlier work this paper cites.
“Dataset Condensation with Gradient Matching”
Bo Zhao, Konda Mopuri and Hakan Bilen · 2006
Earlier work this paper cites.
“Flexible Dataset Distillation: Learn Labels Instead of Images”
Ondrej Bohdal, Yongxin Yang and Timothy Hospedales · 2006
Earlier work this paper cites.
“Dataset Condensation with Gradient Matching”
Bo Zhao, Konda Mopuri and Hakan Bilen · 2006
Earlier work this paper cites.
“Dataset Meta-Learning from Kernel Ridge-Regression”
Timothy Nguyen, Zhourong Chen and Jaehoon Lee · 2011
Earlier work this paper cites.
“Dataset Meta-Learning from Kernel Ridge-Regression”
Timothy Nguyen, Zhourong Chen and Jaehoon Lee · 2011
Earlier work this paper cites.
“Do deep nets really need to be deep?”
Jimmy Ba and R Caruana · 2013
Earlier work this paper cites.
“Do deep nets really need to be deep?”
Jimmy Ba and R Caruana · 2013
Earlier work this paper cites.
“Distilling the Knowledge in a Neural Network”
Geoffrey Hinton, Oriol Vinyals and Jeff Dean · 2015
Earlier work this paper cites.
“Distilling the Knowledge in a Neural Network”
Geoffrey Hinton, Oriol Vinyals and Jeff Dean · 2015
Earlier work this paper cites.
“TorchVision: PyTorch’s Computer Vision library”
Torchvision Maintainers and Contributors · 2016
Earlier work this paper cites.
“TorchVision: PyTorch’s Computer Vision library”
Torchvision Maintainers and Contributors · 2016
Earlier work this paper cites.
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba and Alexei Efros · 2018
Earlier work this paper cites.
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba and Alexei Efros · 2018
Earlier work this paper cites.
“Revisiting Knowledge Distillation via label smoothing regularization”
Li Yuan et al · 2020
Earlier work this paper cites.
“Revisiting Knowledge Distillation via label smoothing regularization”
Li Yuan et al · 2020
Earlier work this paper cites.
“Knowledge Distillation: A Survey”
Jianping Gou, Baosheng Yu, Stephen Maybank and Dacheng Tao · 2021
Cited alongside, same era.
“A Fast Knowledge Distillation Framework for Visual Recognition”
Zhiqiang Shen and Eric Xing · 2021
Cited alongside, same era.
“Soft-Label Dataset Distillation and Text Dataset Distillation”
Ilia Sucholutsky and Matthias Schonlau · 2021
Cited alongside, same era.
“Dataset Condensation with Differentiable Siamese Augmentation”
Bo Zhao and Hakan Bilen · 2021
Cited alongside, same era.
“Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective”
Helong Zhou et al · 2021
Cited alongside, same era.
“Knowledge Distillation: A Survey”
“Generalizing dataset distillation via deep generative prior”
George Cazenavette et al · 2023
Later among the works it cites.
“Embarassingly Simple Dataset Distillation”
Yunzhen Feng, Ramakrishna Vedantam and Julia Kempe · 2023
Later among the works it cites.
Suriya Gunasekar et al · 2023
Later among the works it cites.
“Towards lossless Dataset Distillation via difficulty-aligned trajectory matching”
Ziyao Guo et al · 2023
Later among the works it cites.
Noveen Sachdeva and Julian McAuley · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jianping Gou, Baosheng Yu, Stephen Maybank and Dacheng Tao · 2021
Cited alongside, same era.
“A Fast Knowledge Distillation Framework for Visual Recognition”
Zhiqiang Shen and Eric Xing · 2021
Cited alongside, same era.
“Soft-Label Dataset Distillation and Text Dataset Distillation”
Ilia Sucholutsky and Matthias Schonlau · 2021
Cited alongside, same era.
“Dataset Condensation with Differentiable Siamese Augmentation”
Bo Zhao and Hakan Bilen · 2021
Cited alongside, same era.
“Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective”
Helong Zhou et al · 2021
Cited alongside, same era.
“Dataset Distillation by Matching Training Trajectories”
George Cazenavette et al · 2022
Cited alongside, same era.
“Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory”
Justin Cui, Ruochen Wang, Si Si and Cho-Jui Hsieh · 2022
Cited alongside, same era.
Later among the works it cites.
“Generalized large-scale data condensation via various backbone and Statistical Matching”
Shitong Shao et al · 2023
Later among the works it cites.
“Dataset Distillation in Large Data Era”
Zeyuan Yin and Zhiqiang Shen · 2023
Later among the works it cites.
“Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective”
Zeyuan Yin, Eric Xing and Zhiqiang Shen · 2023
Later among the works it cites.
“Dataset Condensation with Distribution Matching”
Bo Zhao and Hakan Bilen · 2023
Later among the works it cites.
“Generalizing dataset distillation via deep generative prior”
George Cazenavette et al · 2023
Later among the works it cites.
“Embarassingly Simple Dataset Distillation”
Yunzhen Feng, Ramakrishna Vedantam and Julia Kempe · 2023
Later among the works it cites.
Suriya Gunasekar et al · 2023
Later among the works it cites.
“Towards lossless Dataset Distillation via difficulty-aligned trajectory matching”
Ziyao Guo et al · 2023
Later among the works it cites.
Noveen Sachdeva and Julian McAuley · 2023
Later among the works it cites.
“Generalized large-scale data condensation via various backbone and Statistical Matching”
Shitong Shao et al · 2023
Later among the works it cites.
“Dataset Distillation in Large Data Era”
Zeyuan Yin and Zhiqiang Shen · 2023
Later among the works it cites.
“Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective”
Zeyuan Yin, Eric Xing and Zhiqiang Shen · 2023
Later among the works it cites.
“Dataset Condensation with Distribution Matching”
Bo Zhao and Hakan Bilen · 2023
Later among the works it cites.
“Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone”
Marah Abdin et al · 2024
Closest in time.
“Phi-2: The surprising power of small language models” Accessed: 2024-5-21, https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/
Alyssa Hughes · 2024
Closest in time.
“Distributional Dataset Distillation with Subtask Decomposition”
Tian Qin, Zhiwei Deng and David Alvarez-Melis · 2024
Closest in time.
“What is Dataset Distillation Learning?”
William Yang, Ye Zhu, Zhiwei Deng and Olga Russakovsky · 2024
Closest in time.
“Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone”
Marah Abdin et al · 2024
Closest in time.
“Phi-2: The surprising power of small language models” Accessed: 2024-5-21, https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/
Alyssa Hughes · 2024
Closest in time.
“Distributional Dataset Distillation with Subtask Decomposition”
Tian Qin, Zhiwei Deng and David Alvarez-Melis · 2024
Closest in time.
“What is Dataset Distillation Learning?”
William Yang, Ye Zhu, Zhiwei Deng and Olga Russakovsky · 2024
Closest in time.