Image captioning aims to describe visual content in natural language. As 'a picture is worth a thousand words', there could be various correct descriptions for an image. However, with maximum likelihood estimation as the training objective, the captioning model is penalized whenever its prediction mismatches with the label. For instance, when the model predicts a word expressing richer semantics than the label, it will be penalized and optimized to prefer more concise expressions, referred to as conciseness optimization. In contrast, predictions that are more concise than labels lead to richness optimization. Such conflicting optimization directions could eventually result in the model generating general descriptions. In this work, we introduce Semipermeable MaxImum Likelihood Estimation (SMILE), which allows richness optimization while blocking conciseness optimization, thus encouraging the model to generate longer captions with more details. Extensive experiments on two mainstream image captioning datasets MSCOCO and Flickr30K demonstrate that SMILE significantly enhances the descriptiveness of generated captions. We further provide in-depth investigations to facilitate a better understanding of how SMILE works.

翻译：图像字幕旨在用自然语言描述视觉内容。由于“一图胜千言”，同一幅图像可能存在多种正确的描述方式。然而，当以最大似然估计作为训练目标时，只要模型的预测与标签不匹配就会受到惩罚。例如，当模型预测的词语比标签表达更丰富的语义时，它会被惩罚并优化为倾向于更简洁的表达，这称为简洁性优化。相反，当预测比标签更简洁时，则导致丰富度优化。这种相互冲突的优化方向最终可能导致模型生成泛化的描述。在本工作中，我们提出了半透最大似然估计（SMILE），该方法允许丰富度优化同时阻断简洁性优化，从而鼓励模型生成更长的、包含更多细节的字幕。在两个主流图像字幕数据集MSCOCO和Flickr30K上的大量实验证明，SMILE显著提升了生成字幕的描述性。我们进一步开展了深入研究，以促进对SMILE工作机制的更好理解。