Due to increasing popularity and strict performance requirements, online games have become a workload of interest for the performance engineering community. One of the most popular types of online games is the Minecraft-like Game (MLG), in which players can terraform the environment. The most popular MLG, Minecraft, provides not only entertainment, but also educational support and social interaction, to over 130 million people world-wide. MLGs currently support their many players by replicating isolated instances that support each only up to a few hundred players under favorable conditions. In practice, as we show here, the real upper limit of supported players can be much lower. In this work, we posit that performance variability is a key cause for the lack of scalability in MLGs. We propose a novel operational model for MLGs and use it to design the first benchmark that focuses on MLG performance variability, defining specialized workloads, metrics, and processes. We conduct real-world benchmarking of MLGs and find environment-based workloads and cloud deployment to be significant sources of performance variability: peak-latency degrades sharply to 20.7 times the arithmetic mean, and exceeds by a factor of 7.4 the performance requirements. We derive actionable insights for game-developers, game-operators, and other stakeholders to tame performance variability.

翻译：随着在线游戏日益普及且性能要求日趋严格，其已成为性能工程领域关注的重要工作负载。类《我的世界》游戏（MLG）作为最受欢迎的在线游戏类型之一，允许玩家改造游戏环境。其中最流行的MLG游戏《我的世界》不仅为全球超1.3亿用户提供娱乐功能，还兼具教育支持与社交互动价值。当前MLG通过复制独立实例来支持众多玩家，在理想条件下每个实例仅能支撑数百人并发。然而实际研究显示，系统可承载的真实玩家上限可能远低于此。本文提出，性能变异性是制约MLG可扩展性的关键因素。我们构建了MLG的新型操作模型，并据此设计首个聚焦MLG性能变异性的基准测试，定义专项工作负载、指标体系与测试流程。通过真实环境下的MLG基准测试，我们发现环境交互型工作负载与云端部署是性能变异性的重要诱因：峰值延迟剧烈恶化至算术平均值的20.7倍，超过性能需求阈值的7.4倍。我们为游戏开发者、运营商及其他利益相关者提炼出可操作的优化建议，以有效管控性能变异性。