We have witnessed remarkable advances in LLM reasoning capabilities with the advent of DeepSeek-R1. However, much of this progress has been fueled by the abundance of internet question-answer (QA) pairs, a major bottleneck going forward, since such data is limited in scale and concentrated mainly in domains like mathematics. In contrast, other sciences such as physics lack large-scale QA datasets to effectively train reasoning-capable models. In this work, we show that physics simulators can serve as a powerful alternative source of supervision for training LLMs for physical reasoning. We generate random scenes in physics engines, create synthetic question-answer pairs from simulated interactions, and train LLMs using reinforcement learning on this synthetic data. Our models exhibit zero-shot sim-to-real transfer to real-world physics benchmarks: for example, training solely on synthetic simulated data improves performance on IPhO (International Physics Olympiad) problems by 5-10 percentage points across model sizes. These results demonstrate that physics simulators can act as scalable data generators, enabling LLMs to acquire deep physical reasoning skills beyond the limitations of internet-scale QA data. Code available at: https://sim2reason.github.io/.

翻译：随着DeepSeek-R1的出现，我们在大型语言模型推理能力方面见证了显著的进步。然而，这种进步很大程度上得益于互联网上丰富的问答（QA）数据对，这构成了未来发展的一个主要瓶颈，因为此类数据规模有限且主要集中在数学等特定领域。相比之下，物理学等其他科学领域缺乏大规模问答数据集来有效训练具备推理能力的模型。在这项工作中，我们表明物理模拟器可以作为训练大型语言模型进行物理推理的强大替代监督源。我们在物理引擎中生成随机场景，从模拟交互中创建合成问答对，并利用强化学习在这些合成数据上训练大型语言模型。我们的模型展现出零样本模拟到现实的迁移能力，可应用于现实世界的物理基准测试：例如，仅使用合成模拟数据进行训练，即可在不同规模模型上将国际物理奥林匹克（IPhO）问题的表现提升5-10个百分点。这些结果表明，物理模拟器可以作为可扩展的数据生成器，使大型语言模型能够超越互联网规模问答数据的限制，获得深层次的物理推理能力。代码地址：https://sim2reason.github.io/。