Vector approximate nearest neighbor search (ANNS) underpins search engines, recommendation systems, and advertising services. Recent advances in ANNS indexes make CPU a cost-effective choice for serving million-scale, in-memory vector search, yet per-core throughput remains constrained by memory access latency of vector reading and the compute intensity of distance evaluations in production deployments. With the growing scale of the business and advances in hardware, modern CCD-based multi-core CPUs have been widely deployed for high throughput in our services. However, we find that simply increasing core counts does not yield optimal performance scaling. To improve the efficiency of more cores from the CCD-based architecture, we analyze the distributions of real-world requests in our production environments. We observe high access locality in vector search in our online services and low cache utilization, resulting from overlooking the multi-chiplet nature of CCD based CPUs. Hence, we propose a workload- and hardware-aware thread orchestration framework at CCD-level that (i) provides a uniform interface for both inter-query parallel HNSW search and intra-query parallel IVF search, (ii) achieves cache-friendly and workload-adaptive mapping of task dispatching, and (iii) employs CCD-aware task stealing to address load imbalance. Applied to real production workloads from search, recommendation, and advertising services of Xiaohongshu (RedNote), our approach delivers up to 3.7x higher throughput and 30-90% reductions in P50 and P999 latency. In detail, compared with the original framework, the cache-miss ratio decreases by 6-30%, and the total CPU stall is reduced by 20-80%.

翻译：向量近似最近邻搜索（ANNS）是搜索引擎、推荐系统和广告服务的核心支撑技术。近年来ANNS索引的进步使得CPU成为服务百万级内存向量搜索的经济高效选择，但在生产部署中，每核吞吐量仍受限于向量读取的内存访问延迟和距离计算的算力强度。随着业务规模扩展与硬件进步，基于CCD的现代多核CPU已广泛应用于我们服务中以实现高吞吐量。然而，我们发现简单增加核心数并不能实现最优性能扩展。为提升CCD架构下多核效率，我们分析了生产环境中真实请求的分布特征。在在线服务中观察到向量搜索存在高访问局部性，同时由于忽略CCD多芯粒特性导致缓存利用率低下。为此，我们提出一种CCD级别的负载与硬件感知线程编排框架，该框架能够：(i) 为跨查询并行HNSW搜索与查询内并行IVF搜索提供统一接口，(ii) 实现缓存友好且负载自适应的任务调度映射，(iii) 采用CCD感知的任务窃取机制解决负载不均衡问题。将该方法应用于小红书（RedNote）搜索、推荐与广告服务的真实生产负载后，吞吐量提升最高达3.7倍，P50和P999延迟降低30-90%。具体而言，与原始框架相比，缓存缺失率降低6-30%，CPU总停顿时间减少20-80%。