Existing Vision-Language Pretraining (VLP) methods have achieved remarkable improvements across a variety of vision-language tasks, confirming their effectiveness in capturing coarse-grained semantic correlations. However, their capability for fine-grained understanding, which is critical for many nuanced vision-language applications, remains limited. Prevailing VLP models often overlook the intricate distinctions in expressing different modal features and typically depend on the similarity of holistic features for cross-modal interactions. Moreover, these models directly align and integrate features from different modalities, focusing more on coarse-grained general representations, thus failing to capture the nuanced differences necessary for tasks demanding a more detailed perception. In response to these limitations, we introduce Negative Augmented Samples(NAS), a refined vision-language pretraining model that innovatively incorporates NAS to specifically address the challenge of fine-grained understanding. NAS utilizes a Visual Dictionary(VD) as a semantic bridge between visual and linguistic domains. Additionally, it employs a Negative Visual Augmentation(NVA) method based on the VD to generate challenging negative image samples. These samples deviate from positive samples exclusively at the token level, thereby necessitating that the model discerns the subtle disparities between positive and negative samples with greater precision. Comprehensive experiments validate the efficacy of NAS components and underscore its potential to enhance fine-grained vision-language comprehension.

翻译：现有的视觉语言预训练方法已在多种视觉语言任务中取得显著进展，证实了其在捕捉粗粒度语义关联方面的有效性。然而，其细粒度理解能力——这对许多需要精细处理的视觉语言应用至关重要——仍存在局限。主流的视觉语言预训练模型往往忽视不同模态特征表达的细微差异，通常依赖整体特征的相似性进行跨模态交互。此外，这些模型直接对齐和整合不同模态的特征，更侧重于粗粒度的通用表示，因而难以捕捉需要更精细感知的任务所必需的细微差别。针对这些局限性，我们提出负样本增强模型，这是一种改进的视觉语言预训练模型，创新性地引入负样本增强机制以专门应对细粒度理解的挑战。该模型利用视觉词典作为视觉与语言领域之间的语义桥梁。此外，它采用基于视觉词典的负视觉增强方法生成具有挑战性的负样本图像。这些样本仅在标记级别与正样本存在差异，从而要求模型以更高的精度辨别正负样本间的细微差别。综合实验验证了负样本增强模型各组成部分的有效性，并突显了其在提升细粒度视觉语言理解能力方面的潜力。