Learning a feature point detector presents a challenge both due to the ambiguity of the definition of a keypoint and correspondingly the need for a specially prepared ground truth labels for such points. In our work, we address both of these issues by utilizing a combination of a hand-crafted Shi detector and a neural network. We build on the principled and localized keypoints provided by the Shi detector and perform their selection using the keypoint stability score regressed by the neural network - Neural Stability Score (NeSS). Therefore, our method is named Shi-NeSS since it combines the Shi detector and the properties of the keypoint stability score, and it only requires for training sets of images without dataset pre-labeling or the need for reconstructed correspondence labels. We evaluate Shi-NeSS on HPatches, ScanNet, MegaDepth and IMC-PT, demonstrating state-of-the-art performance and good generalization on downstream tasks.

翻译：学习特征点检测器面临双重挑战：一方面，特征点定义本身存在歧义性；另一方面，需要为此类点专门制备真实的标签数据。本研究通过融合手工设计的Shi检测器与神经网络，同时解决上述两个问题。我们以Shi检测器提供的具有原理基础且定位精确的特征点为基础，利用神经网络回归得到的特征点稳定性评分——神经稳定性评分（NeSS）进行特征点选择。该方法结合了Shi检测器与特征点稳定性评分的特性，故命名为Shi-NeSS。该方案仅需训练图像集，无需数据集预标注或重建对应标签。我们在HPatches、ScanNet、MegaDepth和IMC-PT数据集上评估了Shi-NeSS，实验表明其在下游任务中具有先进的性能表现与良好的泛化能力。