Existing Optimal Transport (OT) methods mainly derive the optimal transport plan/matching under the criterion of transport cost/distance minimization, which may cause incorrect matching in some cases. In many applications, annotating a few matched keypoints across domains is reasonable or even effortless in annotation burden. It is valuable to investigate how to leverage the annotated keypoints to guide the correct matching in OT. In this paper, we propose a novel KeyPoint-Guided model by ReLation preservation (KPG-RL) that searches for the optimal matching (i.e., transport plan) guided by the keypoints in OT. To impose the keypoints in OT, first, we propose a mask-based constraint of the transport plan that preserves the matching of keypoint pairs. Second, we propose to preserve the relation of each data point to the keypoints to guide the matching. The proposed KPG-RL model can be solved by Sinkhorn's algorithm and is applicable even when distributions are supported in different spaces. We further utilize the relation preservation constraint in the Kantorovich Problem and Gromov-Wasserstein model to impose the guidance of keypoints in them. Meanwhile, the proposed KPG-RL model is extended to the partial OT setting. Moreover, we deduce the dual formulation of the KPG-RL model, which is solved using deep learning techniques. Based on the learned transport plan from dual KPG-RL, we propose a novel manifold barycentric projection to transport source data to the target domain. As applications, we apply the proposed KPG-RL model to the heterogeneous domain adaptation and image-to-image translation. Experiments verified the effectiveness of the proposed approach.

翻译：现有最优传输（OT）方法主要基于传输成本/距离最小化准则推导最优传输计划/匹配，这可能导致某些情况下的错误匹配。在许多应用中，跨域标注少量匹配的关键点是合理的，甚至在标注负担上可以忽略不计。研究如何利用已标注关键点指导OT中的正确匹配具有重要价值。本文提出一种新颖的基于关系保持的关键点引导模型（KPG-RL），该模型通过OT中的关键点引导搜索最优匹配（即传输计划）。为在OT中施加关键点约束，首先，我们提出了基于掩码的传输计划约束，用于保持关键点对的匹配关系。其次，我们提出保持每个数据点与关键点之间的关系以引导匹配。所提出的KPG-RL模型可通过Sinkhorn算法求解，且适用于分布位于不同空间的情况。进一步地，我们在Kantorovich问题和Gromov-Wasserstein模型中引入关系保持约束，以实现关键点引导。同时，所提出的KPG-RL模型被扩展到部分OT设置中。此外，我们推导了KPG-RL模型的对偶形式，并采用深度学习技术求解。基于从对偶KPG-RL学得的传输计划，我们提出一种新颖的流形重心投影方法，用于将源数据迁移到目标域。在应用层面，我们将所提出的KPG-RL模型应用于异构域自适应和图像到图像翻译任务。实验验证了所提方法的有效性。