Fetching the paper…

Reading the bibliography…

Dataset Distillation: A Comprehensive Review · Around

2023

Dataset Distillation: A Comprehensive Review

Yu, Ruonan, Liu, Songhua, Wang, Xinchao

Original
PDF
Post on X

Understand

Recent success of deep learning is largely attributed to the sheer amount of data used for training deep neural networks.Despite the unprecedented success, the massive data, unfortunately, significantly increases the burden on storage and transmission and further gives rise to a cumbersome model training process.

Besides, relying on the raw data for training \emph{per se} yields concerns about privacy and copyright.
To alleviate these shortcomings, dataset distillation~(DD), also known as dataset condensation (DC), was introduced and has recently attracted much research attention in the community.
Given an original dataset, DD aims to derive a much smaller dataset containing synthetic samples, based on which the trained models yield performance comparable with those trained on the original dataset.

Built on

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998
1998
Earlier work this paper cites.
J. Platt et al. , “Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods,” Advances in large margin classifiers , vol. 10, no. 3, pp. 61–74, 1999
1999
Earlier work this paper cites.
A. Narayanan and V. Shmatikov, “Robust de-anonymization of large sparse datasets,” in 2008 IEEE Symposium on Security and Privacy (sp 2008) . IEEE, 2008, pp. 111–125
2008
Earlier work this paper cites.
M. Welling, “Herding dynamical weights to learn,” in Proceedings of the 26th Annual International Conference on Machine Learning , 2009, pp. 1121–1128
2009
Earlier work this paper cites.
A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009
2009
Earlier work this paper cites.
S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , 2017, pp. 2001–2010
2010
Earlier work this paper cites.
X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proceedings of the thirteenth international conference on artificial intelligence and statistics . JMLR Workshop and Conference Proceedings, 2010, pp. 249–256
2010
Earlier work this paper cites.
E. R. Weitzman, L. Kaci, and K. D. Mandl, “Sharing medical data for health research: the early personal health record experience,” Journal of medical Internet research , vol. 12, no. 2, p. e1356, 2010
2010
Earlier work this paper cites.
D. Feldman, M. Faulkner, and A. Krause, “Scalable training of mixture models via coresets,” Advances in neural information processing systems , vol. 24, 2011
2011
Earlier work this paper cites.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” 2011
2011
Earlier work this paper cites.
Y. Chen, M. Welling, and A. Smola, “Super-samples from kernel herding,” arXiv preprint arXiv:1203.3472 , 2012
Original
2012
Earlier work this paper cites.
J. Snoek, H. Larochelle, and R. P. Adams, “Practical bayesian optimization of machine learning algorithms,” Advances in neural information processing systems , vol. 25, 2012
2012
Earlier work this paper cites.
R. M. Neal, Bayesian learning for neural networks . Springer Science & Business Media, 2012, vol. 118
2012
Earlier work this paper cites.
J. A. Konstan and J. Riedl, “Recommender systems: from algorithms to user experience,” User modeling and user-adapted interaction , vol. 22, no. 1, pp. 101–123, 2012
2012
Earlier work this paper cites.
I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y. Bengio, “An empirical investigation of catastrophic forgetting in gradient-based neural networks,” arXiv preprint arXiv:1312.6211 , 2013
Original
2013
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114 , 2013
Original
2013
Earlier work this paper cites.
J. Bergstra, D. Yamins, and D. Cox, “Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures,” in International conference on machine learning . PMLR, 2013, pp. 115–123
2013
Earlier work this paper cites.
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, “Fitnets: Hints for thin deep nets,” arXiv preprint arXiv:1412.6550 , 2014
Original
2014
Earlier work this paper cites.
J. Ba and R. Caruana, “Do deep nets really need to be deep?” Advances in neural information processing systems , vol. 27, 2014
2014
Earlier work this paper cites.
M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784 , 2014
Original
2014
Earlier work this paper cites.
J. J. Pfeiffer III, S. Moreno, T. La Fond, J. Neville, and B. Gallagher, “Attributed graph models: Modeling network structure with correlated attributes,” in Proceedings of the 23rd international conference on World wide web , 2014, pp. 831–842
2014
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
Original
2014
Earlier work this paper cites.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2014, pp. 580–587
2014
Earlier work this paper cites.
D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473 , 2014
Original
2014
Earlier work this paper cites.
D. Maclaurin, D. Duvenaud, and R. Adams, “Gradient-based hyperparameter optimization through reversible learning,” in International conference on machine learning . PMLR, 2015, pp. 2113–2122
2015
Earlier work this paper cites.
R. Shokri and V. Shmatikov, “Privacy-preserving deep learning,” in Proceedings of the 22nd ACM SIGSAC conference on computer and communications security , 2015, pp. 1310–1321
2015
Earlier work this paper cites.
G. Hinton, O. Vinyals, J. Dean et al. , “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , vol. 2, no. 7, 2015
Original
2015
Earlier work this paper cites.
O. Bachem, M. Lucic, and A. Krause, “Coresets for nonparametric estimation-the case of dp-means,” in International Conference on Machine Learning . PMLR, 2015, pp. 209–217
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 1026–1034
2015
Earlier work this paper cites.
M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in Proceedings of the 22nd ACM SIGSAC conference on computer and communications security , 2015, pp. 1322–1333
2015
Earlier work this paper cites.
F. O. Isinkaye, Y. O. Folajimi, and B. A. Ojokoh, “Recommendation systems: Principles, methods and evaluation,” Egyptian informatics journal , vol. 16, no. 3, pp. 261–273, 2015
2015
Earlier work this paper cites.
Y. Le and X. Yang, “Tiny imagenet visual recognition challenge,” CS 231N , vol. 7, no. 7, p. 3, 2015
2015
Earlier work this paper cites.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al. , “Imagenet large scale visual recognition challenge,” International journal of computer vision , vol. 115, no. 3, pp. 211–252, 2015
2015
Earlier work this paper cites.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 3431–3440
2015
Earlier work this paper cites.
D. Amodei, S. Ananthanarayanan, R. Anubhai, J. Bai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, Q. Cheng, G. Chen et al. , “Deep speech 2: End-to-end speech recognition in english and mandarin,” in International conference on machine learning . PMLR, 2016, pp. 173–182
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
S. Zagoruyko and N. Komodakis, “Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer,” arXiv preprint arXiv:1612.03928 , 2016
Original
2016
Earlier work this paper cites.
J. Konečnỳ, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” arXiv preprint arXiv:1610.02527 , 2016
Original
2016
Earlier work this paper cites.
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907 , 2016
Original
2016
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM , vol. 60, no. 6, pp. 84–90, 2017
2017
Earlier work this paper cites.
S. You, C. Xu, C. Xu, and D. Tao, “Learning from multiple teacher networks,” in Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2017, pp. 1285–1294
2017
Earlier work this paper cites.
R. G. Lopes, S. Fenu, and T. Starner, “Data-free knowledge distillation for deep neural networks,” arXiv preprint arXiv:1710.07535 , 2017
Original
2017
Earlier work this paper cites.
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861 , 2017
Original
2017
Earlier work this paper cites.
L. Franceschi, M. Donini, P. Frasconi, and M. Pontil, “Forward and reverse gradient-based hyperparameter optimization,” in International Conference on Machine Learning . PMLR, 2017, pp. 1165–1173
2017
Earlier work this paper cites.
O. Sener and S. Savarese, “Active learning for convolutional neural networks: A core-set approach,” arXiv preprint arXiv:1708.00489 , 2017
Original
2017
Earlier work this paper cites.
C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 , ser. Proceedings of Machine Learning Research, D. Precup and Y. W. Teh, Eds., vol. 70. PMLR, 2017, pp. 1126–1135. [Online]. Available: http://proceedings.mlr.press/v70/finn17a.html
2017
Earlier work this paper cites.
J. Lee, Y. Bahri, R. Novak, S. S. Schoenholz, J. Pennington, and J. Sohl-Dickstein, “Deep neural networks as gaussian processes,” arXiv preprint arXiv:1711.00165 , 2017
Original
2017
Earlier work this paper cites.
T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks with cutout,” arXiv preprint arXiv:1708.04552 , 2017
Original
2017
Earlier work this paper cites.
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al. , “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences , vol. 114, no. 13, pp. 3521–3526, 2017
2017
Earlier work this paper cites.
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics . PMLR, 2017, pp. 1273–1282
2017
Earlier work this paper cites.
R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE symposium on security and privacy (SP) . IEEE, 2017, pp. 3–18
2017
Earlier work this paper cites.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083 , 2017
Original
2017
Earlier work this paper cites.
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” arXiv preprint arXiv:1710.10903 , 2017
Original
2017
Earlier work this paper cites.
H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747 , 2017
Original
2017
Earlier work this paper cites.
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE transactions on pattern analysis and machine intelligence , vol. 40, no. 4, pp. 834–848, 2017
2017
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Earlier work this paper cites.

Similar

T. Wang, J.-Y. Zhu, A. Torralba, and A. A. Efros, “Dataset distillation,” arXiv preprint arXiv:1811.10959 , 2018
Original
2018
Cited alongside, same era.
X. Wang, R. Zhang, Y. Sun, and J. Qi, “Kdgan: Knowledge distillation with generative adversarial networks,” Advances in neural information processing systems , vol. 31, 2018
2018
Cited alongside, same era.
X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely efficient convolutional neural network for mobile devices,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 6848–6856
2018
Cited alongside, same era.
I. Radosavovic, P. Dollár, R. Girshick, G. Gkioxari, and K. He, “Data distillation: Towards omni-supervised learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 4119–4128
2018
Cited alongside, same era.

Then

B. Zhao and H. Bilen, “Dataset condensation with differentiable siamese augmentation,” in International Conference on Machine Learning . PMLR, 2021, pp. 12 674–12 685
2021
Later among the works it cites.
Bo Zhao and Hakan Bilen, “Dataset condensation with distribution matching,” CoRR , vol. abs/2110.04181, 2021
Original
2021
Later among the works it cites.
F. Wiewel and B. Yang, “Condensed composite memory continual learning,” in 2021 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2021, pp. 1–8
2021
Later among the works it cites.
J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision , vol. 129, no. 6, pp. 1789–1819, 2021
2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

L. Franceschi, P. Frasconi, S. Salzo, R. Grazzi, and M. Pontil, “Bilevel programming for hyperparameter optimization and meta-learning,” in International Conference on Machine Learning . PMLR, 2018, pp. 1568–1577

2018

Cited alongside, same era.

A. Jacot, F. Gabriel, and C. Hongler, “Neural tangent kernel: Convergence and generalization in neural networks,” Advances in neural information processing systems , vol. 31, 2018

2018

Cited alongside, same era.

B. Tran, J. Li, and A. Madry, “Spectral signatures in backdoor attacks,” Advances in neural information processing systems , vol. 31, 2018

2018

Cited alongside, same era.

P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner et al. , “Relational inductive biases, deep learning, and graph networks,” arXiv preprint arXiv:1806.01261 , 2018

Original

2018

Cited alongside, same era.

Z. Ying, J. You, C. Morris, X. Ren, W. Hamilton, and J. Leskovec, “Hierarchical graph representation learning with differentiable pooling,” Advances in neural information processing systems , vol. 31, 2018

2018

Cited alongside, same era.

T. Elsken, J. H. Metzen, and F. Hutter, “Neural architecture search: A survey,” The Journal of Machine Learning Research , vol. 20, no. 1, pp. 1997–2017, 2019

2019

Cited alongside, same era.

W. Park, D. Kim, Y. Lu, and M. Cho, “Relational knowledge distillation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 3967–3976

2019

Cited alongside, same era.

J. Liu, D. Wen, H. Gao, W. Tao, T.-W. Chen, K. Osa, and M. Kato, “Knowledge representing: efficient, sparse representation of prior knowledge for knowledge distillation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops , 2019, pp. 0–0

2019

Cited alongside, same era.

K. Killamsetty, X. Zhao, F. Chen, and R. Iyer, “Retrieve: Coreset selection for efficient and robust semi-supervised learning,” Advances in Neural Information Processing Systems , vol. 34, pp. 14 488–14 501, 2021

2021

Later among the works it cites.

K. Killamsetty, D. Sivasubramanian, G. Ramakrishnan, and R. Iyer, “Glister: Generalization based data subset selection for efficient and robust learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 9, 2021, pp. 8110–8118

2021

Later among the works it cites.

I. Sucholutsky and M. Schonlau, “Soft-label dataset distillation and text dataset distillation,” in 2021 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2021, pp. 1–8

2021

Later among the works it cites.

P. Buzzega, M. Boschini, A. Porrello, and S. Calderara, “Rethinking experience replay: a bag of tricks for continual learning,” in 2020 25th International Conference on Pattern Recognition (ICPR) . IEEE, 2021, pp. 2180–2187

2021

Later among the works it cites.

Y. Liu, B. Schiele, and Q. Sun, “Adaptive aggregation networks for class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 2544–2553

2021

Later among the works it cites.

Y. Li, X. Lyu, N. Koren, L. Lyu, B. Li, and X. Ma, “Neural attention distillation: Erasing backdoor triggers from deep neural networks,” arXiv preprint arXiv:2101.05930 , 2021

2021

Later among the works it cites.

H. Kwon, “Defending deep neural networks against backdoor attack by using de-trigger autoencoder,” IEEE Access , 2021

2021

Later among the works it cites.

J. Hayase, W. Kong, R. Somani, and S. Oh, “Spectre: defending against backdoor attacks using robust statistics,” arXiv preprint arXiv:2104.11315 , 2021

2021

Later among the works it cites.

W. Jin, L. Zhao, S. Zhang, Y. Liu, J. Tang, and N. Shah, “Graph condensation for graph neural networks,” arXiv preprint arXiv:2110.07580 , 2021

2021

Later among the works it cites.

S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep learning–based text classification: a comprehensive review,” ACM Computing Surveys (CSUR) , vol. 54, no. 3, pp. 1–40, 2021

2021

Later among the works it cites.

Y. Li and W. Li, “Data distillation for text classification,” arXiv preprint arXiv:2104.08448 , 2021

Original

2021

Later among the works it cites.

P. Chen, S. Liu, H. Zhao, and J. Jia, “Distilling knowledge via knowledge review,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 5008–5017

2021

Later among the works it cites.

R. Kumar, W. Wang, J. Kumar, T. Yang, A. Khan, W. Ali, and I. Ali, “An integration of blockchain and ai for secure data sharing and detection of ct images for the hospitals,” Computerized Medical Imaging and Graphics , vol. 87, p. 101812, 2021

2021

Later among the works it cites.

A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125 , 2022

2022

Later among the works it cites.

C. Chen, Y. Zhang, J. Fu, X. Liu, and M. Coates, “Bidirectional learning for offline infinite-width model-based optimization,” in Thirty-Sixth Conference on Neural Information Processing Systems , 2022. [Online]. Available: https://openreview.net/forum?id=_j8yVIyp27Q

2022

Later among the works it cites.

T. Dong, B. Zhao, and L. Lyu, “Privacy for free: How does dataset condensation help privacy?” arXiv preprint arXiv:2206.00240 , 2022

Original

2022

Later among the works it cites.

Y. Zhou, E. Nezhadarya, and J. Ba, “Dataset distillation using neural feature regression,” arXiv preprint arXiv:2206.00719 , 2022

2022

Later among the works it cites.

A. Rosasco, A. Carta, A. Cossu, V. Lomonaco, and D. Bacciu, “Distilled replay: Overcoming forgetting through synthetic samples,” in International Workshop on Continual Semi-Supervised Learning . Springer, 2022, pp. 104–117

2022

Later among the works it cites.

M. Sangermano, A. Carta, A. Cossu, and D. Bacciu, “Sample condensation in online continual learning,” in 2022 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2022, pp. 01–08

2022

Later among the works it cites.

Y. Xiong, R. Wang, M. Cheng, F. Yu, and C.-J. Hsieh, “Feddm: Iterative distribution matching for communication-efficient federated learning,” arXiv preprint arXiv:2207.09653 , 2022

2022

Later among the works it cites.

R. Song, D. Liu, D. Z. Chen, A. Festag, C. Trinitis, M. Schulz, and A. Knoll, “Federated learning via decentralized dataset distillation in resource-constrained edge environments,” arXiv preprint arXiv:2208.11311 , 2022

2022

Later among the works it cites.

P. Liu, X. Yu, and J. T. Zhou, “Meta knowledge condensation for federated learning,” arXiv preprint arXiv:2209.14851 , 2022

2022

Later among the works it cites.

S. Hu, J. Goetz, K. Malik, H. Zhan, Z. Liu, and Y. Liu, “Fedsynth: Gradient compression via synthetic data in federated learning,” arXiv preprint arXiv:2204.01273 , 2022

2022

Later among the works it cites.

R. Pi, W. Zhang, Y. Xie, J. Gao, X. Wang, S. Kim, and Q. Chen, “Dynafed: Tackling client data heterogeneity with global dynamics,” arXiv preprint arXiv:2211.10878 , 2022

2022

Later among the works it cites.

G. Fang, K. Mo, X. Wang, J. Song, S. Bei, H. Zhang, and M. Song, “Up to 100x faster data-free knowledge distillation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 6, 2022, pp. 6597–6604

2022

Later among the works it cites.

B. Zhao and H. Bilen, “Synthesizing informative training samples with gan,” arXiv preprint arXiv:2204.07513 , 2022

2022

Later among the works it cites.

H. B. Lee, D. B. Lee, and S. J. Hwang, “Dataset condensation with latent space knowledge factorization and sharing,” arXiv preprint arXiv:2208.10494 , 2022

2022

Later among the works it cites.

J. Cui, R. Wang, S. Si, and C.-J. Hsieh, “DC-BENCH: Dataset condensation benchmark,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) , 2022

2022

Later among the works it cites.

S. Lee, S. Chun, S. Jung, S. Yun, and S. Yoon, “Dataset condensation with contrastive signals,” in Proceedings of the International Conference on Machine Learning (ICML) , 2022, pp. 12 352–12 364

2022

Later among the works it cites.

Z. Jiang, J. Gu, M. Liu, and D. Z. Pan, “Delving into effective gradient matching for dataset condensation,” arXiv preprint arXiv:2208.00311 , 2022

2022

Later among the works it cites.

G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu, “Dataset distillation by matching training trajectories,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 4750–4759

2022

Later among the works it cites.

J.-H. Kim, J. Kim, S. J. Oh, S. Yun, H. Song, J. Jeong, J.-W. Ha, and H. O. Song, “Dataset condensation via efficient synthetic-data parameterization,” arXiv preprint arXiv:2205.14959 , 2022

2022

Later among the works it cites.

Z. Deng and O. Russakovsky, “Remember the past: Distilling datasets into addressable memories for neural networks,” arXiv preprint arXiv:2206.02916 , 2022

2022

Later among the works it cites.

N. Loo, R. Hasani, A. Amini, and D. Rus, “Efficient dataset distillation using random feature approximation,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) , 2022

2022

Later among the works it cites.

L. Zhang, J. Zhang, B. Lei, S. Mukherjee, X. Pan, B. Zhao, C. Ding, Y. Li, and X. Dongkuan, “Accelerating dataset distillation via model augmentation,” arXiv preprint arXiv:2212.06152 , 2022

2022

Later among the works it cites.

Li, Guang and Togo, Ren and Ogawa, Takahiro and Haseyama, Miki, “Dataset distillation using parameter pruning,” arXiv preprint arXiv:2209.14609 , 2022

Original

2022

Later among the works it cites.

J. Cui, R. Wang, S. Si, and C.-J. Hsieh, “Scaling up dataset distillation to imagenet-1k with constant memory,” arXiv preprint arXiv:2211.10586 , 2022

2022

Later among the works it cites.

J. Du, Y. Jiang, V. T. F. Tan, J. T. Zhou, and H. Li, “Minimizing the accumulated trajectory error to improve dataset distillation,” arXiv preprint arXiv:2211.11004 , 2022

2022

Later among the works it cites.

K. Wang, B. Zhao, X. Peng, Z. Zhu, S. Yang, S. Wang, G. Huang, H. Bilen, X. Wang, and Y. You, “Cafe: Learning to condense dataset by aligning features,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 12 196–12 205

2022

Later among the works it cites.

S. Liu, K. Wang, X. Yang, J. Ye, and X. Wang, “Dataset distillation via factorization,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) , 2022

2022

Later among the works it cites.

D. Chen, R. Kerkouche, and M. Fritz, “Private set generation with discriminative information,” arXiv preprint arXiv:2211.04446 , 2022

2022

Later among the works it cites.

N. Tsilivis, J. Su, and J. Kempe, “Can we achieve robustness from data alone?” arXiv preprint arXiv:2207.11727 , 2022

2022

Later among the works it cites.

Y. Wu, X. Li, F. Kerschbaum, H. Huang, and H. Zhang, “Towards robust dataset learning,” 2022. [Online]. Available: https://arxiv.org/abs/2211.10752

2022

Later among the works it cites.

W. Jin, X. Tang, H. Jiang, Z. Li, D. Zhang, J. Tang, and B. Yin, “Condensing graphs via one-step gradient matching,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2022, pp. 720–730

2022

Later among the works it cites.

M. Liu, S. Li, X. Chen, and L. Song, “Graph condensation via receptive field distribution matching,” arXiv preprint arXiv:2206.13697 , 2022

2022

Later among the works it cites.

N. Sachdeva, M. P. Dhaliwal, C.-J. Wu, and J. McAuley, “Infinite recommendation networks: A data-centric approach,” arXiv preprint arXiv:2206.02626 , 2022

2022

Later among the works it cites.

C. Li, M. Lin, Z. Ding, N. Lin, Y. Zhuang, Y. Huang, X. Ding, and L. Cao, “Knowledge condensation distillation,” in European Conference on Computer Vision . Springer, 2022, pp. 19–35

2022

Later among the works it cites.

Li, Guang and Togo, Ren and Ogawa, Takahiro and Haseyama, Miki, “Dataset distillation for medical dataset sharing,” arXiv preprint arXiv:2209.14603 , 2022

2022

Later among the works it cites.

Li, Guang and Togo, Ren and Ogawa, Takahiro and Haseyama, Miki, “Compressed gastric image generation based on soft-label dataset distillation for medical data sharing,” Computer Methods and Programs in Biomedicine , vol. 227, p. 107189, 2022

2022

Later among the works it cites.

G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu, “Wearable imagenet: Synthesizing tileable textures via dataset distillation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 2278–2282

2022

Later among the works it cites.

Y. Chen, Z. Wu, Z. Shen, and J. Jia, “Learning from designers: Fashion compatibility analysis via dataset distillation,” in 2022 IEEE International Conference on Image Processing (ICIP) . IEEE, 2022, pp. 856–860

2022

Later among the works it cites.

Y. Liu, Z. Li, M. Backes, Y. Shen, and Y. Zhang, “Backdoor attacks against dataset distillation,” arXiv preprint arXiv:2301.01197 , 2023

Original

2023

Closest in time.