This paper examines the problems of severe image-text misalignment and high redundancy in the widely-used large-scale Vision-Language Pre-Training (VLP) datasets. To address these issues, we propose an efficient and straightforward Vision-Language learning algorithm called TL;DR, which aims to compress the existing large VLP data into a small, high-quality set. Our approach consists of two major steps. First, a codebook-based encoder-decoder captioner is developed to select representative samples. Second, a new caption is generated to complement the original captions for selected samples, mitigating the text-image misalignment problem while maintaining uniqueness. As the result, TL;DR enables us to reduce the large dataset into a small set of high-quality data, which can serve as an alternative pre-training dataset. This algorithm significantly speeds up the time-consuming pretraining process. Specifically, TL;DR can compress the mainstream VLP datasets at a high ratio, e.g., reduce well-cleaned CC3M dataset from 2.82M to 0.67M ($\sim$24\%) and noisy YFCC15M from 15M to 2.5M ($\sim$16.7\%). Extensive experiments with three popular VLP models over seven downstream tasks show that VLP model trained on the compressed dataset provided by TL;DR can perform similar or even better results compared with training on the full-scale dataset. The code will be made available at \url{https://github.com/showlab/data-centric.vlp}.

翻译：本文研究了广泛使用的大规模视觉语言预训练（VLP）数据集中存在的严重图像-文本错位和高冗余问题。为解决这些问题，我们提出了一种高效且直接的视觉语言学习算法，称为TL;DR，旨在将现有的大规模VLP数据压缩为小型高质量数据集。我们的方法包含两个主要步骤。首先，开发了一个基于码本的编码器-解码器字幕生成器，用于选择代表性样本。其次，为所选样本生成新的标题以补充原始标题，在缓解图文错位问题的同时保持唯一性。因此，TL;DR使我们能够将大型数据集缩减为小型高质量数据集，该数据集可作为替代预训练数据集。该算法显著加速了耗时的预训练过程。具体而言，TL;DR能够以高压缩比压缩主流VLP数据集，例如，将已良好清洗的CC3M数据集从2.82M减少至0.67M（约24%），将含噪的YFCC15M从15M减少至2.5M（约16.7%）。在七个下游任务上使用三种流行VLP模型进行的广泛实验表明，在TL;DR提供的压缩数据集上训练的VLP模型与在全量数据集上训练相比，能够获得相似甚至更好的结果。代码将在\url{https://github.com/showlab/data-centric.vlp}上公开。