Measuring structured object understanding in vision foundation models remains challenging due to inconsistent evaluation protocols and limited part-level supervision. Semantic correspondence (SC) evaluates this capability by testing whether object parts can be matched across instances and categories under large variations in appearance, viewpoint, and geometry. To enable a systematic SC evaluation, we introduce SOCO, a new benchmark for Semantic Object Correspondence that introduces a taxonomy of correspondence types and provides consistent, functionally meaningful keypoint annotations across 100 categories and over 1M correspondence pairs. In addition, SOCO includes keypoint language descriptions, enabling the evaluation of large vision-language models (LVLMs) and their fine-grained part-level understanding. Comprehensive experiments reveal that (i) vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories and only partially capture object-part position, (ii) LVLMs are stronger at text-prompted part localization than at visual-reference cross-image matching, exposing a gap between language-grounded localization and fine-grained visual correspondence, and (iii) correspondence performance predicts performance on dense downstream tasks, including segmentation, tracking, 3D pose estimation, and 3D detection, more strongly than ImageNet classification. Together, these findings position SOCO as a benchmark for structured, part-level representation quality in vision and multimodal foundation models.

翻译：由于评估协议不一致且部件级监督有限，衡量视觉基础模型中的结构化对象理解能力仍具挑战性。语义对应关系通过测试对象部件在跨实例和跨类别条件下，能否在表观、视角与几何形态发生显著变化时实现匹配，来评估这一能力。为实现系统性的语义对应关系评估，我们提出SOCO——一个面向语义对象对应关系的新基准。该基准构建了对应关系类型学分类体系，并在100个类别、超过100万个对应关系对上提供了语义一致且功能意义明确的关键点标注。此外，SOCO还包含关键点语言描述，支持对大型视觉语言模型及其细粒度部件级理解能力的评估。综合实验表明：(i) 视觉基础骨架编码了强语义结构，但在相关类别间迁移对应关系时表现欠佳，且仅能部分捕获对象部件的位置信息；(ii) 大型视觉语言模型在文本提示驱动的部件定位任务中强于基于视觉参考的跨图像匹配任务，这揭示了语言锚定的定位能力与细粒度视觉对应关系之间存在鸿沟；(iii) 与ImageNet分类相比，对应关系性能更能预测密集下游任务的表现，包括分割、跟踪、3D姿态估计和3D检测。综合而言，这些发现将SOCO定位为衡量视觉与多模态基础模型中结构化部件级表征质量的基准。