The prevalence of vector similarity search in modern machine learning applications and the continuously changing nature of data processed by these applications necessitate efficient and effective index maintenance techniques for vector search indexes. Designed primarily for static workloads, existing vector search indexes degrade in search quality and performance as the underlying data is updated unless costly index reconstruction is performed. To address this, we introduce Ada-IVF, an incremental indexing methodology for Inverted File (IVF) indexes. Ada-IVF consists of 1) an adaptive maintenance policy that decides which index partitions are problematic for performance and should be repartitioned and 2) a local re-clustering mechanism that determines how to repartition them. Compared with state-of-the-art dynamic IVF index maintenance strategies, Ada-IVF achieves an average of 2x and up to 5x higher update throughput across a range of benchmark workloads.

翻译：现代机器学习应用中向量相似性搜索的普及，以及这些应用处理数据持续变化的特性，对向量搜索索引的高效有效维护技术提出了迫切需求。现有向量搜索索引主要针对静态工作负载设计，当底层数据更新时，除非执行代价高昂的索引重建，否则搜索质量和性能会逐渐下降。为解决这一问题，我们提出了Ada-IVF——一种面向倒排文件（IVF）索引的增量索引方法。Ada-IVF包含：1）自适应维护策略，用于判定哪些索引分区存在性能问题并需要重新划分；2）局部重聚类机制，用于确定如何对这些分区进行重新划分。与最先进的动态IVF索引维护策略相比，Ada-IVF在一系列基准工作负载上实现了平均2倍、最高5倍的更新吞吐量提升。