Distill Gold from Massive Ores: Bi-level Data Pruning towards Efficient Dataset Distillation

Understand

Data-efficient learning has garnered significant attention, especially given the current trend of large multi-modal models.

• Recently, dataset distillation has become an effective approach by synthesizing data samples that are essential for network training.
• However, it remains to be explored which samples are essential for the dataset distillation process itself.
• In this work, we study the data efficiency and selection for the dataset distillation task.