Similarity search is a central problem in domains such as information management and retrieval or data analysis. Many similarity search algorithms are designed or specifically adapted to metric distances. Thus, they are unsuitable for alternatives like the cosine distance, which has become quite common, for example, with embeddings and in text mining. This paper presents GDASC (General Distributed Approximate Similarity search with Clustering), a general framework for distributed approximate similarity search that accepts arbitrary distances. This framework can build a multilevel index structure, by selecting a clustering algorithm, the number of prototypes in each cluster and any arbitrary distance function. As a result, this framework effectively overcomes the limitation of using metric distances and can address situations involving cosine similarity or other non-standard similarity measures. Experimental results using k-medoids clustering in GDASC with real datasets confirm the applicability of this approach for approximate similarity search, improving the performance of extant algorithms for this purpose.

翻译：相似性搜索是信息管理与检索或数据分析等领域的核心问题。许多相似性搜索算法专为度量距离设计或特别适配，因此不适用于余弦距离等替代方案——这类距离在嵌入表示和文本挖掘等领域已变得相当普遍。本文提出GDASC（基于聚类的通用分布式近似相似性搜索），这是一种支持任意距离的分布式近似相似性搜索通用框架。该框架能够通过选择聚类算法、设定每个簇的质心数量以及采用任意距离函数，构建多层次索引结构。由此，该框架有效克服了使用度量距离的限制，能够处理涉及余弦相似性或其他非标准相似性度量的场景。在真实数据集上使用k-medoids聚类进行GDASC实验的结果证实了该方法在近似相似性搜索中的适用性，提升了现有相关算法的性能。