Memory performance is often the main bottleneck in modern computing systems. In recent years, researchers have attempted to scale the memory wall by leveraging new technology such as CXL, HBM, and in- and near-memory processing. Developers optimizing for such hardware need to understand how target applications perform to fully take advantage of these systems. Existing software and hardware performance introspection techniques are ill-suited for this purpose due to one or more of the following factors: coarse-grained measurement, inability to offer data needed to debug key issues, high runtime overhead, and hardware dependence. The heightened integration between compute and memory in many proposed systems offers an opportunity to extend compiler support for this purpose. We have developed Examem, a memory performance introspection framework based on the LLVM compiler infrastructure. Examem supports developer annotated regions in code, allowing for targeted instrumentation of kernels. Examem supports hardware performance counters when available, in addition to software instrumentation. It statically records information about the instruction mix of the code and adds dynamic instrumentation to produce estimated memory bandwidth for an instrumented region at runtime. This combined approach keeps runtime overhead low while remaining accurate, with a geomean overhead under 10% and a geomean byte accuracy of 93%. Finally, our instrumentation is performed using an LLVM IR pass, which is target agnostic, and we have applied it to four ISAs.

翻译：内存性能通常是现代计算系统的主要瓶颈。近年来，研究者尝试通过利用CXL、HBM以及内存内与近内存处理等新技术来突破内存墙。为这类硬件进行优化的开发者需要了解目标应用程序的实际表现，以充分发挥此类系统的潜力。现有的软硬件性能检测技术由于以下一个或多个因素而难以满足此需求：测量粒度粗、无法提供调试关键问题所需数据、运行时开销高以及硬件依赖性。许多新型系统中计算与内存的高度集成为扩展编译器支持提供了契机。我们开发了Examem——一个基于LLVM编译器基础设施的内存性能检测框架。Examem支持开发者在代码中标注特定区域，实现对核心代码的定向检测。除软件检测外，Examem在可用时还支持硬件性能计数器。该框架静态记录指令组合信息，并通过动态检测在运行时生成被检测区域的预估内存带宽。这种混合方法在保持高精度的同时实现了低运行时开销，其几何平均开销低于10%，字节级精度几何平均值达93%。最后，我们的检测通过目标无关的LLVM IR通道实现，并已成功应用于四种指令集架构。