Canonical surface mapping generalizes keypoint detection by assigning each pixel of an object to a corresponding point in a 3D template. Popularised by DensePose for the analysis of humans, authors have since attempted to apply the concept to more categories, but with limited success due to the high cost of manual supervision. In this work, we introduce SHIC, a method to learn canonical maps without manual supervision which achieves better results than supervised methods for most categories. Our idea is to leverage foundation computer vision models such as DINO and Stable Diffusion that are open-ended and thus possess excellent priors over natural categories. SHIC reduces the problem of estimating image-to-template correspondences to predicting image-to-image correspondences using features from the foundation models. The reduction works by matching images of the object to non-photorealistic renders of the template, which emulates the process of collecting manual annotations for this task. These correspondences are then used to supervise high-quality canonical maps for any object of interest. We also show that image generators can further improve the realism of the template views, which provide an additional source of supervision for the model.

翻译：规范表面映射通过将物体的每个像素分配到三维模板中的对应点，从而推广了关键点检测。该方法最初由DensePose在人体分析中推广，随后研究者尝试将其应用于更多类别，但由于人工监督成本高昂而成效有限。本文提出SHIC——一种无需人工监督即可学习规范映射的方法，在多数类别上取得了优于监督方法的结果。我们的核心思想是利用DINO和Stable Diffusion等基础计算机视觉模型，这些模型具有开放特性，因而对自然类别拥有优异的先验知识。SHIC通过使用基础模型的特征，将估计图像到模板对应关系的问题转化为预测图像到图像对应关系的问题。该转化通过将物体图像与模板的非真实感渲染图像进行匹配来实现，这模拟了为此任务收集人工标注的过程。这些对应关系随后被用于监督任意目标物体的高质量规范映射。我们还证明图像生成器能进一步提升模板视图的真实感，从而为模型提供额外的监督来源。