Dataset Distillation: Principles, Methods, and Applications

Dataset distillation is the process of creating a small, synthetic or selected dataset that captures the essential information of a much larger dataset, such that models trained on the distilled set perform nearly as well as those trained on the full data. Also known as dataset condensation or coreset selection, this technique addresses a fundamental tension in modern machine learning: the need for large-scale data versus the practical constraints of storage, computation, and accessibility.