This paper examines the problems of severe image-text misalignment and high redundancy in the widely-used large-scale Vision-Language Pre-Training (VLP) datasets. To address these issues, we propose an efficient and straightforward Vision-Language learning algorithm called TL;DR, which aims to compress the existing large VLP data into a small, high-quality set. Our approach consists of two major steps. First, a codebook-based encoder-decoder captioner is developed to select representative samples. Second, a new caption is generated to complement the original captions for selected samples, mitigating the text-image misalignment problem while maintaining uniqueness. As the result, TL;DR enables us to reduce the large dataset into a small set of high-quality data, which can serve as an alternative pre-training dataset. This algorithm significantly speeds up the time-consuming pretraining process. Specifically, TL;DR can compress the mainstream VLP datasets at a high ratio, e.g., reduce well-cleaned CC3M dataset from 2.82M to 0.67M ($\sim$24\%) and noisy YFCC15M from 15M to 2.5M ($\sim$16.7\%). Extensive experiments with three popular VLP models over seven downstream tasks show that VLP model trained on the compressed dataset provided by TL;DR can perform similar or even better results compared with training on the full-scale dataset. The code will be made available at \url{https://github.com/showlab/datacentric.vlp}.

翻译：本文研究了广泛使用的大规模视觉-语言预训练（VLP）数据集中存在的严重图文错配与高冗余性问题。为解决这些问题，我们提出了一种名为TL;DR的高效简洁视觉-语言学习算法，旨在将现有大规模VLP数据压缩为小型高质量数据集。该方法包含两个主要步骤：首先，开发基于码本的编码器-解码器描述生成器来选取代表性样本；其次，为所选样本生成新描述以补充原始描述，在缓解图文错配问题的同时保持样本独特性。由此，TL;DR可将大型数据集缩减为可用于替代预训练的小型高质量数据集。该算法显著加速了耗时的预训练过程。具体而言，TL;DR能以高压缩比处理主流VLP数据集，例如将经过良好清洗的CC3M数据集从282万缩减至67万（约24%），将含噪的YFCC15M数据集从1500万缩减至250万（约16.7%）。在三个主流VLP模型和七个下游任务上的大量实验表明，基于TL;DR压缩数据集训练的VLP模型性能与全量数据集训练相当甚至更优。代码将发布于\url{https://github.com/showlab/datacentric.vlp}。