Image Captioning is a current research task to describe the image content using the objects and their relationships in the scene. To tackle this task, two important research areas converge, artificial vision, and natural language processing. In Image Captioning, as in any computational intelligence task, the performance metrics are crucial for knowing how well (or bad) a method performs. In recent years, it has been observed that classical metrics based on n-grams are insufficient to capture the semantics and the critical meaning to describe the content in an image. Looking to measure how well or not the set of current and more recent metrics are doing, in this article, we present an evaluation of several kinds of Image Captioning metrics and a comparison between them using the well-known MS COCO dataset. The metrics were selected from the most used in prior works, they are those based on $n$-grams as BLEU, SacreBLEU, METEOR, ROGUE-L, CIDEr, SPICE, and those based on embeddings, such as BERTScore and CLIPScore. For this, we designed two scenarios; 1) a set of artificially build captions with several qualities, and 2) a comparison of some state-of-the-art Image Captioning methods. Interesting findings were found trying to answer the questions: Are the current metrics helping to produce high-quality captions? How do actual metrics compare to each other? What are the metrics really measuring?

翻译：图像描述是一项当前的研究任务，旨在利用场景中的对象及其关系来描述图像内容。为解决这一任务，两大研究领域——计算机视觉和自然语言处理——在此交汇。如同任何计算智能任务，性能度量对于评估方法的优劣至关重要。近年来，人们发现基于n-gram的传统度量不足以捕捉用于描述图像内容的语义和关键含义。为衡量当前及较新度量方法的性能，本文对多种图像描述度量进行了评估，并使用著名的MS COCO数据集进行了比较。所选度量来自以往研究中最常用的指标，包括基于n-gram的BLEU、SacreBLEU、METEOR、ROGUE-L、CIDEr、SPICE，以及基于嵌入的度量如BERTScore和CLIPScore。为此，我们设计了两种场景：1）一套人工构建的、具有不同质量的描述文本；2）对若干最先进的图像描述方法进行比较。研究发现了一些有趣的结果，试图回答以下问题：当前度量是否有助于生成高质量描述？实际度量之间如何相互比较？度量真正衡量的是什么？