Similarity join finds all pairs of close points within a given distance threshold. Many similarity join methods have been proposed, but they are usually not efficient on high-dimensional space due to the curse of dimensionality and data-unawareness. We investigate the possibility of using metric space Bloom filter (MSBF), a family of data structures checking if a query point has neighbors in a multi-dimensional space, to speed up similarity join. However, there are several challenges when applying MSBF to similarity join, including excessive information loss, data-unawareness and hard constraint on the distance metric. In this paper, we propose Xling, a generic framework to build a learning-based metric space filter with any existing regression model, aiming at accurately predicting whether a query point has enough number of neighbors. The framework provides a suite of optimization strategies to further improve the prediction quality based on the learning model, which has demonstrated significantly higher prediction quality than existing MSBF. We also propose XJoin, one of the first filter-based similarity join methods, based on Xling. By predicting and skipping those queries without enough neighbors, XJoin can effectively reduce unnecessary neighbor searching and therefore it achieves a remarkable acceleration. Benefiting from the generalization capability of deep learning models, XJoin can be easily transferred onto new dataset (in similar distribution) without re-training. Furthermore, Xling is not limited to being applied in XJoin, instead, it acts as a flexible plugin that can be inserted to any loop-based similarity join methods for a speedup.

翻译：相似连接旨在找出给定距离阈值内的所有相近点对。尽管已有多种相似连接方法被提出，但由于维度灾难和数据不可知性，这些方法在高维空间中通常效率低下。我们研究了利用度量空间布鲁姆过滤器（MSBF，一种用于判断查询点在多维空间中是否存在邻域点的数据结构族）来加速相似连接的可行性。然而，将MSBF应用于相似连接面临多重挑战，包括信息过度丢失、数据不可知性以及距离度量的硬性约束。本文提出Xling——一个通用框架，可基于任意现有回归模型构建学习型度量空间过滤器，旨在精确预测查询点是否包含足够数量的邻域点。该框架提供了一套基于学习模型的优化策略来进一步提升预测质量，实验证明其预测质量显著优于现有MSBF。基于Xling，我们提出了XJoin——首批基于过滤器的相似连接方法之一。通过预测并跳过那些邻域点不足的查询，XJoin能有效减少不必要的邻域搜索，从而实现显著的加速效果。得益于深度学习模型的泛化能力，XJoin可轻松迁移至（具有相似分布的）新数据集而无需重新训练。此外，Xling的应用不限于XJoin，它可作为灵活插件插入任何基于循环的相似连接方法中实现加速。