Deploying large-scale LLM training and inference with optimal performance is exceptionally challenging due to a complex design space of parallelism strategies, system optimizations, and hardware configurations. Accurate and rapid performance simulation is critical for guiding optimization efforts and system studies by validating "what-if" Hooker Figure hypotheses. To address this, we introduce Charon, a unified, modular, and fine-grained simulator for accurately predicting LLM performance. Experiments show Charon achieves high accuracy across different models and configurations, with an overall prediction error consistently under 5.35%, and even under 3.74% for training with a large-scale GPU cluster. In a practical inference deployment case, Charon discovered a configuration that improved system throughput over an engineering-tuned baseline, demonstrating its significant real-world value.

翻译：部署大规模LLM训练与推理时，由于并行策略、系统优化和硬件配置的复杂设计空间，实现最优性能极具挑战性。准确快速的性能模拟通过验证"假设分析"Hooker图假设，对指导优化工作和系统研究至关重要。为此，我们提出Charon——一个统一、模块化且细粒度的模拟器，用于精确预测LLM性能。实验表明，Charon在不同模型和配置下均能实现高精度，整体预测误差持续低于5.35%，在大型GPU集群训练场景下甚至低于3.74%。在实际推理部署案例中，Charon发现了一种配置方案，使系统吞吐量超过工程调优基线，展现了其重要的实际应用价值。