Open-world text-guided class-agnostic counting (CAC) has emerged as a flexible paradigm for counting arbitrary object classes by using natural language prompts. However, current evaluation protocols primarily focus on standard counting errors within single-category images, overlooking a fundamental requirement: the ability to correctly ground the textual prompt in the visual scene. In this paper, we show that several state-of-the-art CAC models often struggle to determine which object class should be counted based on the given prompt, revealing a misalignment between textual semantics and visual object representations. This limitation leads to spurious counting responses and reduced reliability in real-world scenarios. To systematically address these limitations, we propose a new evaluation framework focused on model robustness and trustworthiness. Our contribution is two-fold: (i) we introduce PrACo++ (Prompt-Aware Counting++), a novel test suite featuring two dedicated evaluation protocols -- the negative-label test and the distractor test -- paired with new specialized metrics; and (ii) we present the MUCCA (MUlti-Category Class-Agnostic counting) evaluation dataset, a new collection of real-world images featuring multiple annotated object categories per scene, unlike existing CAC benchmarks that typically include a single category per image. Our extensive experimental evaluation of 10 state-of-the-art methods shows that, despite strong performance under standard counting metrics, current models exhibit significant weaknesses in understanding and grounding object class descriptions. Finally, we provide a quantitative analysis of how semantic similarity between prompts influences these failures. Overall, our results underscore the need for more semantically grounded architectures and offer a reliable framework for future assessment in open-world text-guided CAC methods.

翻译：开放世界文本引导的类别无关计数（CAC）已成为一种灵活范式，能够通过自然语言提示对任意对象类别进行计数。然而，当前的评估协议主要关注单类别图像内的标准计数误差，忽略了一个基本需求：正确地将文本提示锚定到视觉场景中的能力。在本文中，我们展示了多个最先进的CAC模型往往难以根据给定提示确定应计数的对象类别，揭示了文本语义与视觉对象表征之间的错位。这一局限导致虚假计数响应，并降低了现实场景中的可靠性。为系统性地解决这些问题，我们提出了一种以模型鲁棒性和可信度为重点的新型评估框架。我们的贡献有两方面：（i）我们引入了PrACo++（提示感知计数++），这是一个新的测试套件，包含两个专用评估协议——负标签测试和干扰项测试，并搭配了新的专门化指标；（ii）我们提出了MUCCA（多类别类别无关计数）评估数据集，这是一个新的真实世界图像集合，每个场景包含多个带注释的对象类别，与现有通常每张图像仅包含单一类别的CAC基准不同。我们对10种最先进方法的广泛实验评估表明，尽管在标准计数指标下表现强劲，但当前模型在理解和锚定对象类别描述方面存在显著弱点。最后，我们定量分析了提示间的语义相似性如何影响这些失败。总体而言，我们的结果强调了构建更具语义基础架构的必要性，并为未来开放世界文本引导CAC方法的评估提供了可靠框架。