When key-value (KV) stores use SSDs for storing a large number of items, oftentimes they also require large in-memory data structures including indices and caches to be traversed to reduce IOs. This paper considers offloading most of such data structures from the costly host DRAM to secondary memory whose latency is in the microsecond range, an order of magnitude longer than those of currently available DIMM-mounted or CXL memory devices. While emerging microsecond-latency memory is likely to cost much less than DRAM, it can significantly slow down SSD-based KV stores if naively employed. This paper analyzes and evaluates the impact of microsecond-level memory latency on the KV operation throughput. Our analysis finds that a well-known latency-hiding technique of software prefetching for long-latency memory from user-level threads is effective. The novelty of our analysis lies in modeling how the interplay between prefetching and IO affects performance, from which we derive an equation that well explains the throughput degradation due to long memory latency. The model tells us that the presence of IO significantly enhances the tolerance to memory latency, leading to a finding that SSD-based KV stores can be made latency-tolerant without devising new techniques for microsecond-latency memory. To confirm this, we design a microbenchmark as well as modify existing SSD-based KV stores so that they issue prefetches from user-level threads, and run them while placing most of in-memory data structures on FPGA-based memory with adjustable microsecond latency. The results demonstrate that their KV operation throughputs can be well explained by our model, and the modified KV stores achieve near-DRAM throughputs for up to a memory latency of 5 microseconds. This suggests the possibility that SSD-based KV stores can use microsecond-latency memory as a cost-effective alternative to the host DRAM.

翻译：当键值（KV）存储使用SSD存储海量数据项时，通常还需要维护大型内存数据结构（包括索引与缓存）以支持遍历操作来减少I/O开销。本文探讨将此类数据结构的主体从昂贵的主机DRAM卸载至延迟在微秒量级的二级内存，其延迟比当前可用的DIMM插槽式或CXL内存设备高出一个数量级。尽管新兴的微秒级延迟内存成本可能远低于DRAM，但若直接使用会显著降低基于SSD的KV存储性能。本文系统分析并评估了微秒级内存延迟对KV操作吞吐量的影响。分析发现，针对长延迟内存的软件预取技术——这一广为人知的用户级线程延迟隐藏方法——具有显著效果。本研究的创新点在于建立了预取与I/O交互影响性能的模型，并推导出能准确解释长内存延迟导致吞吐量下降的方程。该模型表明，I/O的存在显著增强了对内存延迟的容忍度，从而揭示出基于SSD的KV存储无需为微秒级延迟内存设计新技术即可实现延迟容忍。为验证此结论，我们设计了微基准测试并改造现有基于SSD的KV存储系统，使其通过用户级线程发起预取请求，同时在具有可调微秒级延迟的FPGA内存上运行主要内存数据结构。实验结果表明，改造后KV存储的吞吐量变化与模型预测高度吻合，且在内存延迟高达5微秒时仍能保持接近DRAM的吞吐性能。这证明基于SSD的KV存储有可能采用微秒级延迟内存作为主机DRAM的高性价比替代方案。