Cycle-level DRAM simulators provide accurate and flexible models for DRAM and memory controller operations and enable research on current and future memory systems. Therefore, they are critical for improving the performance, efficiency, and robustness of DRAM-based memory systems. Ramulator 2.0 (successor of Ramulator) is a highly modular and extensible cycle-accurate DRAM simulator that enables rapid exploration of new ideas in DRAM-based memory systems. A MICRO 2024 best paper runner-up publication, A Mess of Memory System Benchmarking, Simulation and Application Profiling, which we refer to as "the Mess paper," with all three artifact badges awarded (including "Reproducible"), proposes a new benchmark to evaluate real and simulated memory system performance. While doing so, it makes strong negative claims about Ramulator 2.0 and shows unexpected results. In this talk and the associated extended abstract, we demonstrate that these results and claims in the Mess paper are incorrect and are due to configuration and simulator usage errors made in the Mess paper. We describe four best practices to aid users and developers of simulation tools to avoid such issues in the future. We emphasize the importance of contacting simulator authors and developers when unexpected results are observed (especially and importantly before publishing such results), to ensure these simulators are used with correct configurations and as intended. Our investigation also aims to stimulate discussion on artifact evaluation practices and on mechanisms for correcting results and artifacts after publication. To aid future works and reproduction of all our results, we open source all our code and scripts at https://github.com/CMU-SAFARI/Cleaning-up-the-Mess. We refer the reader to our full ISPASS 2026 paper and its artifact for the complete analysis, detailed methodology, and extended results.

翻译：周期级DRAM模拟器为DRAM和内存控制器的操作提供了精确且灵活的模型，并支持对当前及未来内存系统的研究。因此，它们对于提升基于DRAM的内存系统的性能、效率和鲁棒性至关重要。Ramulator 2.0（Ramulator的继任者）是一个高度模块化且可扩展的周期精确DRAM模拟器，能够快速探索基于DRAM的内存系统中的新思路。一篇获得MICRO 2024最佳论文亚军的文章——《内存系统基准测试、模拟与应用性能分析的混乱》（我们简称其为“混乱论文”），该文获得了全部三个工件徽章（包括“可复现”），提出了评估真实与模拟内存系统性能的新基准。在其过程中，该文对Ramulator 2.0提出了强烈的负面论断，并展示了意外结果。在本报告及相关扩展摘要中，我们证明“混乱论文”中的这些结果和论断是错误的，其根源在于“混乱论文”中存在的配置及模拟器使用错误。我们描述了四项最佳实践，以帮助模拟工具的用户和开发者未来避免此类问题。我们强调，在观察到意外结果时（尤其重要的是，在发表此类结果之前），联系模拟器作者和开发者至关重要，以确保这些模拟器以正确的配置和预期方式使用。我们的研究还旨在促进关于工件评估实践以及发表后纠正结果与工件的机制的讨论。为助力未来研究并复现我们所有结果，我们在https://github.com/CMU-SAFARI/Cleaning-up-the-Mess上开源了所有代码和脚本。我们建议读者参阅我们完整的ISPASS 2026论文及其工件，以获取全面分析、详细方法和扩展结果。