Dataset distillation methods promise to reduce large-scale datasets down to significantly smaller sets of (potentially synthetic) training examples, which preserve sufficient information for training a new model from scratch. So far, dataset distillation methods have been developed for image classification. However, with the rise in capabilities of vision-language models (VLMs), and especially given the scale of datasets necessary to train these models, the time is ripe to expand dataset distillation methods beyond image classification. In this work, we take the first steps towards this goal by expanding the idea of trajectory matching to create a distillation method for vision-language datasets. A key challenge is that vision-language datasets do not have a set of discrete classes. To overcome this, our proposed vision-language dataset distillation method jointly distills the image-text pairs in a contrastive formulation. Since there are no existing baselines, we compare our approach to three coreset selection methods (strategic subsampling of the training dataset), which we adapt to the vision-language setting. We demonstrate significant improvements on the challenging Flickr30K and COCO retrieval benchmarks: for example, on Flickr30K, the best coreset selection method selecting 1000 image-text pairs for training achieves only 5.6% image-to-text retrieval accuracy (i.e., recall@1); in contrast, our dataset distillation approach almost doubles that to 9.9% with just 100 (an order of magnitude fewer) training pairs.

翻译：数据集蒸馏方法承诺将大型数据集缩减为显著更小的（可能为合成的）训练样本集，这些样本保留了足够的信息，可用于从头训练新模型。迄今为止，数据集蒸馏方法主要针对图像分类任务而开发。然而，随着视觉-语言模型（VLM）能力的提升，尤其是考虑到训练这些模型所需数据集的规模，将数据集蒸馏方法扩展到图像分类之外的领域恰逢其时。在本工作中，我们通过将轨迹匹配思想扩展到创建视觉-语言数据集的蒸馏方法，迈出了实现这一目标的第一步。一个关键挑战在于视觉-语言数据集不具备离散类别集。为克服这一难题，我们提出的视觉-语言数据集蒸馏方法采用对比学习框架联合蒸馏图像-文本对。由于尚无现有基线，我们将所提方法与三种适用于视觉-语言场景的核心集选择方法（即对训练数据集进行策略性子采样）进行了比较。我们在具有挑战性的Flickr30K和COCO检索基准上取得了显著改进：例如，在Flickr30K上，选择1000个图像-文本对进行训练的最佳核心集选择方法仅达到5.6%的图像到文本检索准确率（即recall@1）；相比之下，我们的数据集蒸馏方法仅用100个训练对（低一个数量级）便几乎翻倍实现了9.9%的准确率。