To meet the demand for large-scale high-dimensional vector approximate nearest neighbor search (ANNS), many graph-based ANNS systems have been widely adopted due to their excellent efficiency-accuracy trade-offs. Nevertheless, in dynamic scenarios involving frequent vector insertions and deletions, existing systems mitigate the overhead by employing batch update strategies, which improve update performance by increasing the batch size. However, excessively increasing the batch size leads to index update delays, which, in turn, cause a significant degradation in query accuracy. This work aims to improve the performance of graph-based ANNS systems in small-batch update scenarios, achieving a balance between update efficiency and query accuracy. We identify two key issues with existing batch update strategies during small-batch updates: (1) significant data waste in disk read/write operations, and (2) frequent triggering of large-scale pruning operations involving high-cost vector computations by the incremental algorithm. To address these issues, we introduce Greator, a disk-based system with a novel graph-based index update method. The core idea of Greator is to accumulate only a small number of vector updates per batch to prevent excessive index degradation, while designing an efficient fine-grained incremental update scheme that reduces data wastage during I/O operations. Additionally, we introduce a lightweight incremental graph repair strategy to reduce pruning operations, thereby minimizing the expensive vector computations. Based on extensive experiments on real-world datasets, Greator can integrate continuous updates faster than the state-of-the-art solutions, achieving up to 4.16X speedup, while maintaining stable index quality to produce low query latency and high query accuracy of approximate vector searches.
翻译:为满足大规模高维向量近似最近邻搜索的需求,众多基于图的近似最近邻搜索系统因其优异的效率-精度平衡特性而被广泛采用。然而,在涉及频繁向量插入与删除的动态场景中,现有系统通过采用批量更新策略来降低开销,即通过增大批量规模来提升更新性能。但过度增大批量规模会导致索引更新延迟,进而造成查询精度的显著下降。本研究旨在提升基于图的近似最近邻搜索系统在小批量更新场景下的性能,实现更新效率与查询精度之间的平衡。我们识别出现有批量更新策略在小批量更新时存在的两个关键问题:(1) 磁盘读写操作中存在显著的数据浪费现象,(2) 增量算法频繁触发涉及高成本向量计算的大规模剪枝操作。为解决这些问题,我们提出了Greator——一个基于磁盘的、采用新型图索引更新方法的系统。Greator的核心思想是每批次仅累积少量向量更新以防止索引质量过度退化,同时设计高效的细粒度增量更新方案以减少I/O操作中的数据浪费。此外,我们引入了一种轻量级的增量图修复策略来减少剪枝操作,从而最小化昂贵的向量计算。基于在真实数据集上的大量实验,Greator能够以优于现有最优方案的速度集成连续更新,实现最高达4.16倍的加速比,同时保持稳定的索引质量以产生低查询延迟和高查询精度的近似向量搜索结果。