Dataset distillation compresses the original data into compact synthetic datasets, reducing training time and storage while retaining model performance, enabling deployment under limited resources. Although recent decoupling-based distillation methods enable dataset distillation at large scale, they continue to face an efficiency gap: optimization-based decoupling methods achieve higher accuracy but demand intensive computation, whereas optimization-free decoupling methods are efficient but sacrifice accuracy. To overcome this trade-off, we propose Exploration--Exploitation Distillation (E$^2$D), a simple, practical method that minimizes redundant computation through an efficient pipeline that begins with full-image initialization to preserve semantic integrity and feature diversity. It then uses a two-phase optimization strategy: an exploration phase that performs uniform updates and identifies high-loss regions, and an exploitation phase that focuses updates on these regions to accelerate convergence. We evaluate E$^2$D on large-scale benchmarks, surpassing the state-of-the-art on ImageNet-1K while being $18\times$ faster, and on ImageNet-21K, our method substantially improves accuracy while remaining $4.3\times$ faster. These results demonstrate that targeted, redundancy-reducing updates, rather than brute-force optimization, bridge the gap between accuracy and efficiency in large-scale dataset distillation. Code is available at https://github.com/ncsu-dk-lab/E2D.

翻译：数据集蒸馏将原始数据压缩为紧凑的合成数据集，在保持模型性能的同时减少训练时间与存储开销，从而实现在有限资源下的部署。尽管近期基于解耦的蒸馏方法已能实现大规模数据集蒸馏，但其仍面临效率瓶颈：基于优化的解耦方法可获得更高精度但计算密集，而无优化的解耦方法虽高效却牺牲了精度。为克服这一权衡，我们提出探索-利用蒸馏（E$^2$D），这是一种简单实用的方法，通过高效流程最小化冗余计算。该方法首先进行全图像初始化以保持语义完整性与特征多样性，随后采用两阶段优化策略：探索阶段执行均匀更新并识别高损失区域，利用阶段则集中对这些区域进行更新以加速收敛。我们在大型基准数据集上评估E$^2$D，其在ImageNet-1K上超越现有最优方法的同时提速18倍；在ImageNet-21K上，本方法在保持4.3倍加速的同时显著提升了精度。这些结果表明，定向且减少冗余的更新策略（而非暴力优化）能够弥合大规模数据集蒸馏中精度与效率之间的鸿沟。代码发布于https://github.com/ncsu-dk-lab/E2D。