We introduce Probabilistic Coordinate Fields (PCFs), a novel geometric-invariant coordinate representation for image correspondence problems. In contrast to standard Cartesian coordinates, PCFs encode coordinates in correspondence-specific barycentric coordinate systems (BCS) with affine invariance. To know \textit{when and where to trust} the encoded coordinates, we implement PCFs in a probabilistic network termed PCF-Net, which parameterizes the distribution of coordinate fields as Gaussian mixture models. By jointly optimizing coordinate fields and their confidence conditioned on dense flows, PCF-Net can work with various feature descriptors when quantifying the reliability of PCFs by confidence maps. An interesting observation of this work is that the learned confidence map converges to geometrically coherent and semantically consistent regions, which facilitates robust coordinate representation. By delivering the confident coordinates to keypoint/feature descriptors, we show that PCF-Net can be used as a plug-in to existing correspondence-dependent approaches. Extensive experiments on both indoor and outdoor datasets suggest that accurate geometric invariant coordinates help to achieve the state of the art in several correspondence problems, such as sparse feature matching, dense image registration, camera pose estimation, and consistency filtering. Further, the interpretable confidence map predicted by PCF-Net can also be leveraged to other novel applications from texture transfer to multi-homography classification.

翻译：我们提出概率坐标场（Probabilistic Coordinate Fields，PCFs），一种用于图像对应问题的新型几何不变坐标表示。与标准笛卡尔坐标不同，PCFs在具有仿射不变性的对应特定重心坐标系（BCS）中对坐标进行编码。为知晓“何时何地信任”编码坐标，我们基于概率网络PCF-Net实现PCFs，该网络将坐标场分布参数化为高斯混合模型。通过联合优化坐标场及其在稠密光流条件下的置信度，PCF-Net可在利用置信度图量化PCFs可靠性时与多种特征描述子协同工作。本工作的一个有趣发现是：学习得到的置信度图收敛于几何一致且语义连贯的区域，这有助于鲁棒的坐标表示。通过将可信坐标输入关键点/特征描述子，我们证明PCF-Net可作为即插即用模块应用于现有依赖对应关系的算法。在室内外数据集上的大量实验表明，精确的几何不变坐标有助于在稀疏特征匹配、稠密图像配准、相机位姿估计和一致性过滤等多个对应问题上达到最新水平。此外，PCF-Net预测的可解释置信度图还可拓展至纹理传递、多单应性分类等新颖应用。