We present a new dataset condensation framework termed Squeeze, Recover and Relabel (SRe$^2$L) that decouples the bilevel optimization of model and synthetic data during training, to handle varying scales of datasets, model architectures and image resolutions for efficient dataset condensation. The proposed method demonstrates flexibility across diverse dataset scales and exhibits multiple advantages in terms of arbitrary resolutions of synthesized images, low training cost and memory consumption with high-resolution synthesis, and the ability to scale up to arbitrary evaluation network architectures. Extensive experiments are conducted on Tiny-ImageNet and full ImageNet-1K datasets. Under 50 IPC, our approach achieves the highest 42.5% and 60.8% validation accuracy on Tiny-ImageNet and ImageNet-1K, outperforming all previous state-of-the-art methods by margins of 14.5% and 32.9%, respectively. Our approach also surpasses MTT in terms of speed by approximately 52$\times$ (ConvNet-4) and 16$\times$ (ResNet-18) faster with less memory consumption of 11.6$\times$ and 6.4$\times$ during data synthesis. Our code and condensed datasets of 50, 200 IPC with 4K recovery budget are available at https://github.com/VILA-Lab/SRe2L.

翻译：我们提出了一种名为“压缩、恢复与重标定”（SRe$^2$L）的新型数据集压缩框架，该框架解耦了训练过程中模型与合成数据的双层优化，以应对不同规模的数据集、模型架构及图像分辨率，实现高效的数据集压缩。所提出的方法展示了跨多种数据集规模的灵活性，并在合成图像任意分辨率、低训练成本与高分辨率合成下的内存消耗控制，以及扩展到任意评估网络架构的能力方面展现出多重优势。我们在Tiny-ImageNet和完整ImageNet-1K数据集上进行了大量实验。在50 IPC下，我们的方法在Tiny-ImageNet和ImageNet-1K上分别达到42.5%和60.8%的最高验证准确率，以14.5%和32.9%的幅度超越所有先前的最优方法。在数据合成过程中，我们的方法速度比MTT快约52倍（ConvNet-4）和16倍（ResNet-18），同时内存消耗分别降低11.6倍和6.4倍。我们的代码及在4000恢复预算下生成的50 IPC和200 IPC压缩数据集已开源至 https://github.com/VILA-Lab/SRe2L。