We propose a simple yet effective and robust method for contrastive captioning: generating discriminative captions that distinguish target images from very similar alternative distractor images. Our approach is built on a pragmatic inference procedure that formulates captioning as a reference game between a speaker, which produces possible captions describing the target, and a listener, which selects the target given the caption. Unlike previous methods that derive both speaker and listener distributions from a single captioning model, we leverage an off-the-shelf CLIP model to parameterize the listener. Compared with captioner-only pragmatic models, our method benefits from rich vision language alignment representations from CLIP when reasoning over distractors. Like previous methods for discriminative captioning, our method uses a hyperparameter to control the tradeoff between the informativity (how likely captions are to allow a human listener to discriminate the target image) and the fluency of the captions. However, we find that our method is substantially more robust to the value of this hyperparameter than past methods, which allows us to automatically optimize the captions for informativity - outperforming past methods for discriminative captioning by 11% to 15% accuracy in human evaluations

翻译：我们提出一种简单、有效且鲁棒的对比性描述生成方法：生成能够将目标图像与高度相似的干扰图像区分开的判别性描述。该方法基于语用推断过程，将描述生成形式化为说话者（生成描述目标图像的可能描述）与听者（根据描述选择目标图像）之间的指涉博弈。不同于以往方法从单一描述模型中同时推导出说话者和听者分布，我们利用现成的CLIP模型参数化听者。相较于仅包含描述者的语用模型，我们的方法在推理干扰图像时受益于CLIP丰富的视觉语言对齐表征。与以往的判别性描述方法类似，我们采用超参数控制描述的信息性（描述能让人类听者区分目标图像的可能性）与流畅性之间的权衡。然而，我们发现该方法对该超参数的鲁棒性显著优于以往方法，从而能够自动优化描述的信息性——在人类评估中，判别性描述的准确率相比以往方法提升11%至15%。