Compute eXpress Link (CXL) has emerged as a key enabler of memory disaggregation for future heterogeneous computing systems to expand memory on-demand and improve resource utilization. However, CXL is still in its infancy stage and lacks commodity products on the market, thus necessitating a reliable system-level simulation tool for research and development. In this paper, we propose CXL-DMSim, an open-source full-system simulator to simulate CXL disaggregated memory systems with high fidelity at a gem5-comparable simulation speed. CXL-DMSim incorporates a flexible CXL memory expander model along with its associated device driver, and CXL protocol support with CXL.io and CXL.mem. It can operate in both app-managed mode and kernel-managed mode, with the latter using a dedicated NUMA-compatible mechanism. The simulator has been rigorously verified against a real hardware testbed with both FPGAand ASIC-based CXL memory devices, which demonstrates the qualification of CXL-DMSim in simulating the characteristics of various CXL memory devices at an average simulation error of 3.4%. The experimental results using LMbench and STREAM benchmarks suggest that the CXL-FPGA memory exhibits a ~2.88x higher latency than local DDR while the CXL-ASIC latency is ~2.18x; CXL-FPGA achieves 45-69% of local DDR memory bandwidth, whereas the number for CXLASIC is 82-83%. The study also reveals that CXL memory can significantly enhance the performance of memory-intensive applications, improved by 23x at most with limited local memory for Viper key-value database and approximately 60% in memorybandwidth-sensitive scenarios such as MERCI. Moreover, the simulator's observability and expandability are showcased with detailed case-studies, highlighting its great potential for research on future CXL-interconnected hybrid memory pool.

翻译：计算快速链路（CXL）已成为未来异构计算系统中内存解聚的关键使能技术，旨在按需扩展内存并提升资源利用率。然而，CXL仍处于发展初期，市场上缺乏商用产品，因此亟需一种可靠的系统级模拟工具以支持研发。本文提出CXL-DMSim，一款开源全系统模拟器，能够在与gem5相当的模拟速度下，以高保真度模拟CXL解聚内存系统。CXL-DMSim集成了一个灵活的CXL内存扩展器模型及其关联设备驱动，并支持包含CXL.io与CXL.mem的CXL协议。它可在应用管理模式与内核管理模式下运行，后者采用专用的NUMA兼容机制。该模拟器已通过基于FPGA和ASIC的CXL内存设备的真实硬件测试平台严格验证，证明CXL-DMSim能够以平均3.4%的模拟误差准确模拟各类CXL内存设备的特性。使用LMbench与STREAM基准测试的实验结果表明：CXL-FPGA内存的延迟约为本地DDR的2.88倍，而CXL-ASIC延迟约为本地DDR的2.18倍；CXL-FPGA可实现本地DDR内存带宽的45-69%，而CXL-ASIC可达82-83%。研究还揭示，CXL内存能显著提升内存密集型应用的性能：对于Viper键值数据库，在本地内存有限的情况下性能最多提升23倍；在MERCI等内存带宽敏感场景中，性能提升约60%。此外，通过详细案例研究展示了模拟器的可观测性与可扩展性，突显了其在未来CXL互连混合内存池研究中的巨大潜力。