The IO500 benchmark has become the community standard for evaluating HPC storage system performance, yet the detailed data contained in its submission packages remains largely unexplored beyond aggregate leaderboard rankings. We present a statistical characterization of 61 IO500 submissions from four competition lists (ISC21 through SC22), examining score distributions, inter-phase correlations, and insights derived from detailed log files that accompany each submission. Our analysis reveals that IO500 scores span four orders of magnitude. Spearman correlation analysis shows strong within-domain clustering for both bandwidth (rs = 0.78 to 0.96) and metadata (rs = 0.89 to 0.98) phases, with the composite sub-scores exhibiting rs = 0.92 at per-node level (Pearson r = 0.53). Log-level analysis uncovers file-system-specific patterns in IOR close-time overhead, straggler behavior during the stonewall wear-down phase, and parallel-find load imbalance that are invisible in aggregate scores. These findings demonstrate that IO500 submission packages constitute a valuable research resource for understanding storage system behavior. The full submission dataset is publicly available at https://github.com/IO500/submission-data, and analysis scripts at https://gitlab-ce.gwdg.de/hpc-team/io500-analysis.
翻译:IO500基准测试已成为评估HPC存储系统性能的社区标准,但其提交包中包含的详细数据在整体排行榜排名之外仍鲜有深入探索。我们对来自四个竞赛榜单(ISC21至SC22)的61份IO500提交进行了统计特征分析,考察了分数分布、阶段间相关性以及随附各提交的详细日志文件中蕴含的见解。分析表明,IO500分数跨越四个数量级。Spearman相关性分析显示,带宽阶段(rs = 0.78至0.96)和元数据阶段(rs = 0.89至0.98)均存在强域内聚类,综合子分数在单节点层面呈现rs = 0.92(Pearson r = 0.53)。日志级分析揭示了IOR关闭开销中的文件系统特定模式、石墙磨损阶段的落后行为以及并行查找负载不均等问题,这些在整体分数中均不可见。这些发现证明,IO500提交包构成了理解存储系统行为的宝贵研究资源。完整的提交数据集公开于https://github.com/IO500/submission-data,分析脚本位于https://gitlab-ce.gwdg.de/hpc-team/io500-analysis。