Past research has proposed numerous hardware prefetching techniques, most of which rely on exploiting one specific type of program context information (e.g., program counter, cacheline address) to predict future memory accesses. These techniques either completely neglect a prefetcher's undesirable effects (e.g., memory bandwidth usage) on the overall system, or incorporate system-level feedback as an afterthought to a system-unaware prefetch algorithm. We show that prior prefetchers often lose their performance benefit over a wide range of workloads and system configurations due to their inherent inability to take multiple different types of program context and system-level feedback information into account while prefetching. In this paper, we make a case for designing a holistic prefetch algorithm that learns to prefetch using multiple different types of program context and system-level feedback information inherent to its design. To this end, we propose Pythia, which formulates the prefetcher as a reinforcement learning agent. For every demand request, Pythia observes multiple different types of program context information to make a prefetch decision. For every prefetch decision, Pythia receives a numerical reward that evaluates prefetch quality under the current memory bandwidth usage. Pythia uses this reward to reinforce the correlation between program context information and prefetch decision to generate highly accurate, timely, and system-aware prefetch requests in the future. Our extensive evaluations using simulation and hardware synthesis show that Pythia outperforms multiple state-of-the-art prefetchers over a wide range of workloads and system configurations, while incurring only 1.03% area overhead over a desktop-class processor and no software changes in workloads. The source code of Pythia can be freely downloaded from https://github.com/CMU-SAFARI/Pythia.

翻译：过去的研究提出了众多硬件预取技术，其中大部分依赖于利用单一类型的程序上下文信息（例如程序计数器、缓存行地址）来预测未来内存访问。这些技术要么完全忽略了预取器对整体系统的不良影响（例如内存带宽占用），要么将系统级反馈作为事后措施附加到未感知系统状态的预取算法中。我们证明，由于固有地无法在预取过程中综合考虑多种不同类型的程序上下文和系统级反馈信息，先前的预取器在多样化工作负载和系统配置下往往会丧失其性能优势。本文中，我们主张设计一种整体性预取算法，该算法能够利用其设计内在的多种不同类型程序上下文和系统级反馈信息进行预取学习。为此，我们提出了Pythia，它将预取器构建为一个强化学习智能体。对于每个需求请求，Pythia会观察多种不同类型的程序上下文信息以做出预取决策。对于每个预取决策，Pythia会收到一个数值奖励，用于评估当前内存带宽使用下的预取质量。Pythia利用此奖励强化程序上下文信息与预取决策之间的关联，从而在未来生成高精度、及时且具备系统感知能力的预取请求。我们通过仿真和硬件综合进行的广泛评估表明，Pythia在多样化工作负载和系统配置下优于多种最先进的预取器，同时在桌面级处理器上仅引入1.03%的面积开销，且无需修改工作负载的软件。Pythia的源代码可从https://github.com/CMU-SAFARI/Pythia 自由下载。