Dataset condensation aims to condense a large dataset with a lot of training samples into a small set. Previous methods usually condense the dataset into the pixels format. However, it suffers from slow optimization speed and large number of parameters to be optimized. When increasing image resolutions and classes, the number of learnable parameters grows accordingly, prohibiting condensation methods from scaling up to large datasets with diverse classes. Moreover, the relations among condensed samples have been neglected and hence the feature distribution of condensed samples is often not diverse. To solve these problems, we propose to condense the dataset into another format, a generative model. Such a novel format allows for the condensation of large datasets because the size of the generative model remains relatively stable as the number of classes or image resolution increases. Furthermore, an intra-class and an inter-class loss are proposed to model the relation of condensed samples. Intra-class loss aims to create more diverse samples for each class by pushing each sample away from the others of the same class. Meanwhile, inter-class loss increases the discriminability of samples by widening the gap between the centers of different classes. Extensive comparisons with state-of-the-art methods and our ablation studies confirm the effectiveness of our method and its individual component. To our best knowledge, we are the first to successfully conduct condensation on ImageNet-1k.

翻译：数据集浓缩旨在将包含大量训练样本的大规模数据集压缩为小型集合。现有方法通常将数据集浓缩为像素格式，但此类方法面临优化速度缓慢和需优化参数数量庞大的问题。当图像分辨率与类别数量增加时，可学习参数会随之增长，导致浓缩方法难以扩展至包含多样类别的大规模数据集。此外，现有方法忽略了浓缩样本间的关联性，导致浓缩样本的特征分布缺乏多样性。为解决上述问题，我们提出将数据集浓缩为新型格式——生成模型。由于生成模型规模随类别数量或图像分辨率增长的稳定性，该新型格式能有效实现大规模数据集的浓缩。同时，我们提出类内损失与类间损失来建模浓缩样本的关联性：类内损失通过将同类样本相互推离以生成更多样化样本，类间损失则通过扩大不同类别中心间距增强样本可判别性。与现有最优方法的广泛对比及消融实验证实了本方法及其各模块的有效性。据我们所知，这是首个成功在ImageNet-1k上完成数据浓缩的研究。