Fetching the paper…
Reading the bibliography…
Dataset distillation or condensation aims to generate a smaller but representative subset from a large dataset, which allows a model to be trained more efficiently, meanwhile evaluating on the original testing data distribution to achieve decent performance.
Curriculum learning
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston · 2009
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Dataset distillation using neural feature regression
Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba · 2009
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Ya Le and Xuan Yang · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
On large-batch training for deep learning: Generalization gap and sharp minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang · 2016
Earlier work this paper cites.
Improved regularization of convolutional neural networks with cutout
Terrance DeVries and Graham W Taylor · 2017
Earlier work this paper cites.
Train longer, generalize better: closing the generalization gap in large batch training of neural networks
Elad Hoffer, Itay Hubara, and Daniel Soudry · 2017
Earlier work this paper cites.
Densely connected convolutional networks
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger · 2017
Earlier work this paper cites.
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros · 2018
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Randaugment: Practical automated data augmentation with a reduced search space
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le · 2020
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020
Earlier work this paper cites.
Momentum contrast for unsupervised visual representation learning
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick · 2020
Earlier work this paper cites.
Designing network design spaces
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár · 2020
Earlier work this paper cites.
Dataset distillation, 2020
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A. Efros · 2020
Earlier work this paper cites.
Dataset condensation with gradient matching
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen · 2020
Earlier work this paper cites.
Dataset distillation with infinitely wide convolutional networks
Timothy Nguyen, Roman Novak, Lechao Xiao, and Jaehoon Lee · 2021
Earlier work this paper cites.
Imagenet-21k pretraining for the masses
Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor · 2021
Cited alongside, same era.
Soft-label dataset distillation and text dataset distillation
Ilia Sucholutsky and Matthias Schonlau · 2021
Cited alongside, same era.
Training data-efficient image transformers & distillation through attention
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou · 2021
Cited alongside, same era.
Dataset condensation with differentiable siamese augmentation
Bo Zhao and Hakan Bilen · 2021
Cited alongside, same era.
Remember the past: Distilling datasets into addressable memories for neural networks
Zhiwei Deng and Olga Russakovsky · 2022
Cited alongside, same era.
Dataset distillation with attention labels for fine-tuning bert
Aru Maekawa, Naoki Kobayashi, Kotaro Funakoshi, and Manabu Okumura · 2023
Closest in time.
Gpt-4 technical report, 2023
OpenAI · 2023
Closest in time.
Datadam: Efficient dataset distillation with attention matching
Ahmad Sajedi, Samir Khaki, Ehsan Amjadian, Lucy Z. Liu, Yuri A. Lawryshyn, and Konstantinos N. Plataniotis · 2023
Closest in time.
On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm
Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin · 2023
Closest in time.
Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective
Zeyuan Yin, Eric Xing, and Zhiqiang Shen · 2023
Closest in time.
Dataset condensation with distribution matching
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Dataset condensation with contrastive signals
Saehyung Lee, Sanghyuk Chun, Sangwon Jung, Sangdoo Yun, and Sungroh Yoon · 2022
Cited alongside, same era.
Efficient dataset distillation using random feature approximation
Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus · 2022
Cited alongside, same era.
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer · 2022
Cited alongside, same era.
A fast knowledge distillation framework for visual recognition
Zhiqiang Shen and Eric Xing · 2022
Cited alongside, same era.
Cafe: Learning to condense dataset by aligning features
Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You · 2022
Cited alongside, same era.
Generalizing dataset distillation via deep generative prior
George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu · 2023
Cited alongside, same era.
Dataset distillation via adversarial prediction matching
Mingyang Chen, Bo Huang, Junda Lu, Bing Li, Yi Wang, Minhao Cheng, and Wei Wang · 2023
Cited alongside, same era.
Bo Zhao and Hakan Bilen · 2023
Closest in time.
One category one prompt: Dataset distillation using diffusion models
Ali Abbasi, Ashkan Shahbazi, Hamed Pirsiavash, and Soheil Kolouri · 2024
Closest in time.
Ameliorate spurious correlations in dataset condensation
Justin Cui, Ruochen Wang, Yuanhao Xiong, and Cho-Jui Hsieh · 2024
Closest in time.
Towards lossless dataset distillation via difficulty-aligned trajectory matching
Ziyao Guo, Kai Wang, George Cazenavette, Hui Li, Kaipeng Zhang, and Yang You · 2024
Closest in time.
Multisize dataset condensation
Yang He, Lingao Xiao, Joey Tianyi Zhou, and Ivor Tsang · 2024
Closest in time.
Curriculum dataset distillation
Zhiheng Ma, Anjia Cao, Funing Yang, and Xing Wei · 2024
Closest in time.
Distributional dataset distillation with subtask decomposition
Tian Qin, Zhiwei Deng, and David Alvarez-Melis · 2024
Closest in time.
Gift: Unlocking full potential of labels in distilled dataset at near-zero cost
Xinyi Shang, Peng Sun, and Tao Lin · 2024
Closest in time.
Frequency domain-based dataset distillation
Donghyeok Shin, Seungjae Shin, and Il-Chul Moon · 2024
Closest in time.
On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm
Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin · 2024
Closest in time.
Dd-robustbench: An adversarial robustness benchmark for dataset distillation
Yifan Wu, Jiawei Du, Ping Liu, Yuewei Lin, Wenqing Cheng, and Wei Xu · 2024
Closest in time.
Towards adversarially robust dataset distillation by curvature regularization
Eric Xue, Yijiang Li, Haoyang Liu, Yifan Shen, and Haohan Wang · 2024
Closest in time.