In this paper, we propose a learning-based image fragment pair-searching and -matching approach to solve the challenging restoration problem. Existing works use rule-based methods to match similar contour shapes or textures, which are always difficult to tune hyperparameters for extensive data and computationally time-consuming. Therefore, we propose a neural network that can effectively utilize neighbor textures with contour shape information to fundamentally improve performance. First, we employ a graph-based network to extract the local contour and texture features of fragments. Then, for the pair-searching task, we adopt a linear transformer-based module to integrate these local features and use contrastive loss to encode the global features of each fragment. For the pair-matching task, we design a weighted fusion module to dynamically fuse extracted local contour and texture features, and formulate a similarity matrix for each pair of fragments to calculate the matching score and infer the adjacent segment of contours. To faithfully evaluate our proposed network, we created a new image fragment dataset through an algorithm we designed that tears complete images into irregular fragments. The experimental results show that our proposed network achieves excellent pair-searching accuracy, reduces matching errors, and significantly reduces computational time. Details, sourcecode, and data are available in our supplementary material.

翻译：本文提出了一种基于学习的图像碎片对搜索与匹配方法，以解决具有挑战性的复原问题。现有研究采用基于规则的方法匹配相似的轮廓形状或纹理，但这类方法难以针对海量数据调整超参数，且计算耗时。为此，我们设计了一种神经网络，能够有效结合邻域纹理与轮廓形状信息，从根本上提升性能。首先，我们利用基于图的网络提取碎片的局部轮廓和纹理特征；针对碎片对搜索任务，采用基于线性变换器的模块整合这些局部特征，并通过对比学习损失编码每个碎片的全局特征；针对碎片对匹配任务，设计加权融合模块动态融合提取的局部轮廓与纹理特征，为每对碎片构建相似度矩阵以计算匹配分数并推断相邻轮廓段。为严格评估所提网络，我们通过自主设计的算法将完整图像撕裂为不规则碎片，构建了新图像碎片数据集。实验结果表明，所提网络实现了优异的碎片对搜索精度，降低了匹配误差，并显著缩短了计算时间。详情、源代码及数据见补充材料。