Modern image captioning models are usually trained with text similarity objectives. However, since reference captions in public datasets often describe the most salient common objects, models trained with text similarity objectives tend to ignore specific and detailed aspects of an image that distinguish it from others. Toward more descriptive and distinctive caption generation, we propose using CLIP, a multimodal encoder trained on huge image-text pairs from web, to calculate multimodal similarity and use it as a reward function. We also propose a simple finetuning strategy of the CLIP text encoder to improve grammar that does not require extra text annotation. This completely eliminates the need for reference captions during the reward computation. To comprehensively evaluate descriptive captions, we introduce FineCapEval, a new dataset for caption evaluation with fine-grained criteria: overall, background, object, relations. In our experiments on text-to-image retrieval and FineCapEval, the proposed CLIP-guided model generates more distinctive captions than the CIDEr-optimized model. We also show that our unsupervised grammar finetuning of the CLIP text encoder alleviates the degeneration problem of the naive CLIP reward. Lastly, we show human analysis where the annotators strongly prefer the CLIP reward to the CIDEr and MLE objectives according to various criteria. Code and Data: https://github.com/j-min/CLIP-Caption-Reward

翻译：现代图像描述模型通常通过文本相似度目标进行训练。然而，由于公开数据集中参考描述往往描述最显著的常见物体，基于文本相似度目标训练的模型倾向于忽略图像中区分于其他图像的具体和细节方面。为了生成更具描述性和独特性的描述，我们提出使用CLIP（一种在海量网络图文对上训练的多模态编码器）计算多模态相似度，并将其作为奖励函数。我们还提出了一种无需额外文本标注的CLIP文本编码器简单微调策略，以改进语法。这完全消除了奖励计算过程中对参考描述的需求。为全面评估描述性描述，我们引入了FineCapEval——一个基于细粒度标准（整体、背景、物体、关系）的描述评估新数据集。在文本到图像检索和FineCapEval上的实验中，所提出的CLIP引导模型生成的描述比CIDEr优化模型更具独特性。我们还证明，对CLIP文本编码器进行的无监督语法微调缓解了朴素CLIP奖励的退化问题。最后，我们展示了人工分析结果：标注者根据多种标准强烈偏好CLIP奖励而非CIDEr和MLE目标。代码与数据：https://github.com/j-min/CLIP-Caption-Reward