Graph Neural Networks (GNNs) with attention have been successfully applied for learning visual feature matching. However, current methods learn with complete graphs, resulting in a quadratic complexity in the number of features. Motivated by a prior observation that self- and cross- attention matrices converge to a sparse representation, we propose ClusterGNN, an attentional GNN architecture which operates on clusters for learning the feature matching task. Using a progressive clustering module we adaptively divide keypoints into different subgraphs to reduce redundant connectivity, and employ a coarse-to-fine paradigm for mitigating miss-classification within images. Our approach yields a 59.7% reduction in runtime and 58.4% reduction in memory consumption for dense detection, compared to current state-of-the-art GNN-based matching, while achieving a competitive performance on various computer vision tasks.

翻译：注意力图神经网络已成功应用于视觉特征匹配学习。然而，现有方法基于完整图进行学习，导致特征数量的二次复杂度。基于自注意力与交叉注意力矩阵收敛于稀疏表示的先前观察，我们提出ClusterGNN——一种基于聚类学习的注意力图神经网络架构。通过渐进式聚类模块，我们自适应地将关键点划分为不同子图以降低冗余连接，并采用粗到细范式减轻图像内误分类。与当前最先进的基于GNN的匹配方法相比，我们的方法在密集检测任务中实现了59.7%的运行时间缩减和58.4%的内存消耗降低，同时在多种计算机视觉任务中保持竞争性性能。