Recent progress in self-supervised representation learning has resulted in models that are capable of extracting image features that are not only effective at encoding image level, but also pixel-level, semantics. These features have been shown to be effective for dense visual semantic correspondence estimation, even outperforming fully-supervised methods. Nevertheless, current self-supervised approaches still fail in the presence of challenging image characteristics such as symmetries and repeated parts. To address these limitations, we propose a new approach for semantic correspondence estimation that supplements discriminative self-supervised features with 3D understanding via a weak geometric spherical prior. Compared to more involved 3D pipelines, our model only requires weak viewpoint information, and the simplicity of our spherical representation enables us to inject informative geometric priors into the model during training. We propose a new evaluation metric that better accounts for repeated part and symmetry-induced mistakes. We present results on the challenging SPair-71k dataset, where we show that our approach demonstrates is capable of distinguishing between symmetric views and repeated parts across many object categories, and also demonstrate that we can generalize to unseen classes on the AwA dataset.

翻译：近期自监督表征学习的进展使得模型能够提取不仅编码图像层面，而且编码像素层面语义的图像特征。这些特征已被证明对密集视觉语义对应估计有效，甚至优于全监督方法。然而，当前的自监督方法在对称性和重复部件等具有挑战性的图像特征面前仍然失败。为解决这些局限性，我们提出了一种新的语义对应估计方法，通过弱几何球面先验，用3D理解补充判别性自监督特征。与更复杂的3D流程相比，我们的模型仅需弱视角信息，且球面表征的简洁性使我们能在训练过程中为模型注入信息丰富的几何先验。我们提出了一种新的评估指标，能更好地解释重复部件和对称性引起的错误。我们在具有挑战性的SPair-71k数据集上展示了结果，表明我们的方法能够区分多个物体类别中的对称视图和重复部件，并证明我们能够在AwA数据集上泛化到未见过的类别。