Reinforcement learning (RL) has been applied to improve large language model (LLM) reasoning, yet the systematic study of how training scales with task difficulty has been hampered by the lack of controlled, scalable environments. We introduce ScaleLogic, a synthetic logical reasoning framework that offers independent control over two axes of difficulty: the depth of the required proof planning (i.e., the horizon) and the expressiveness of the underlying logic. Our proposed framework supports a wide range of logics: from simple implication-only logic ("if-then") towards more expressive first-order reasoning with conjunction ("and"), disjunction ("or"), negation ("not"), and universal quantification ("for all"). Using this framework, we show that the RL training compute $T$ follows a power law with respect to reasoning depth $D$ ($T \propto D^γ$, $R^{2} > 0.99$), and that the scaling exponent $γ$ increases monotonically with logical expressiveness, from $1.04$ to $2.60$. On downstream mathematics and general reasoning benchmarks, more expressive training settings yield both larger performance gains (up to $+10.66$ points) and more compute-efficient transfer compared to less expressive settings, demonstrating that what a model is trained on, not just how much it is trained, shapes downstream transfer. We further show that the power-law relationship holds across multiple RL methods, and curriculum-based training substantially improves scaling efficiency.

翻译：强化学习（RL）已被应用于提升大语言模型（LLM）的推理能力，然而，由于缺乏可控、可扩展的实验环境，关于训练计算量如何随任务难度扩展的系统性研究一直受到阻碍。我们提出了ScaleLogic，这是一个合成逻辑推理框架，能够独立控制两个难度维度：所需证明规划的深度（即步长）以及底层逻辑的表达能力。该框架支持从简单的仅蕴含逻辑（"如果-那么"）到更具表达能力的一阶推理（包含合取"与"、析取"或"、否定"非"及全称量化"所有"）的广泛逻辑类型。利用此框架，我们证明RL训练计算量$T$与推理深度$D$遵循幂律关系（$T \propto D^γ$，$R^{2} > 0.99$），且缩放指数$γ$随逻辑表达能力的增强而单调递增（从$1.04$增至$2.60$）。在下游数学及通用推理基准测试中，相较于表达能力较弱的训练设置，表达能力更强的训练设置不仅带来更大的性能提升（最高达$+10.66$分），还实现了更高效的计算迁移，这表明模型所训练的内容（而不仅仅是训练量）决定了其下游迁移能力。我们进一步证明，该幂律关系在多种RL方法中均成立，且基于课程的学习能显著提高缩放效率。