Mission-critical applications often run "forever" and process large data volumes in real time while demanding low latency. To handle the large state of these applications, modern streaming engines rely on key-value stores and store state on local storage or remotely, but accessing such state inflates latency. As today's engines tightly couple the data path with state I/O, a tuple triggers state access only when it reaches a stateful operator, placing I/O on the critical path and stalling the CPU. However, the keys used to access the state are frequently known earlier in the query plan. Building on this insight, we propose Keyed Prefetching, which decouples the data path from state access by extracting future access keys at upstream operators and proactively staging the corresponding state in memory before tuples arrive. This overlaps I/O with ongoing computation and hides the latency of large-state accesses. We pair Keyed Prefetching with Timestamp-Aware Caching, a cache-eviction policy that jointly manages previously accessed and prefetched entries to use memory efficiently. Together, these techniques reduce latency for long-running, real-time queries without sacrificing throughput.

翻译：关键任务应用通常"永久"运行，实时处理海量数据同时要求低延迟。为处理这些应用的大规模状态，现代流处理引擎依赖键值存储，并将状态存储在本地或远程存储系统中，但访问此类状态会增加延迟。由于当前引擎将数据路径与状态I/O紧密耦合，元组仅当到达有状态算子时才触发状态访问，这使得I/O处于关键路径上并阻塞CPU。然而，用于访问状态的键通常在查询计划的更早期阶段已知。基于这一洞察，我们提出键预取（Keyed Prefetching）技术，通过在上游算子处提取未来访问键，并在元组到达前主动将相应状态预载入内存，从而将数据路径与状态访问解耦。该技术使I/O与正在进行的计算重叠，隐藏了大规模状态访问的延迟。我们将键预取与时间戳感知缓存（Timestamp-Aware Caching）相结合——这是一种联合管理已访问条目和预取条目的缓存逐出策略，以高效利用内存。这些技术协同作用，可在不牺牲吞吐量的情况下降低长时间运行实时查询的延迟。