Human has an incredible ability to effortlessly perceive the viewpoint difference between two images containing the same object, even when the viewpoint change is astonishingly vast with no co-visible regions in the images. This remarkable skill, however, has proven to be a challenge for existing camera pose estimation methods, which often fail when faced with large viewpoint differences due to the lack of overlapping local features for matching. In this paper, we aim to effectively harness the power of object priors to accurately determine two-view geometry in the face of extreme viewpoint changes. In our method, we first mathematically transform the relative camera pose estimation problem to an object pose estimation problem. Then, to estimate the object pose, we utilize the object priors learned from a diffusion model Zero123 to synthesize novel-view images of the object. The novel-view images are matched to determine the object pose and thus the two-view camera pose. In experiments, our method has demonstrated extraordinary robustness and resilience to large viewpoint changes, consistently estimating two-view poses with exceptional generalization ability across both synthetic and real-world datasets. Code will be available at https://github.com/scy639/Extreme-Two-View-Geometry-From-Object-Poses-with-Diffusion-Models.

翻译：人类具有非凡的能力，能够轻松感知包含同一物体的两幅图像之间的视点差异，即使视点变化极其巨大且图像间不存在共同可见区域。然而，这种卓越能力对现有相机姿态估计方法构成了挑战，这些方法由于缺乏用于匹配的局部重叠特征，在面对大视点差异时往往失效。本文旨在有效利用物体先验知识，准确确定极端视点变化下的双视图几何关系。在我们的方法中，首先将相对相机姿态估计问题数学转化为物体姿态估计问题。随后，为估计物体姿态，利用从扩散模型Zero123习得的物体先验，合成物体的新视角图像。通过匹配这些新视角图像来确定物体姿态，进而得到双视图相机姿态。实验表明，本方法对大视点变化具有卓越的鲁棒性和适应性，能在合成数据集和真实世界数据集上持续估计双视图姿态，展现出杰出的泛化能力。代码将开源至https://github.com/scy639/Extreme-Two-View-Geometry-From-Object-Poses-with-Diffusion-Models。