Scaling laws have emerged as a unifying lens for understanding and guiding the training of large language models (LLMs). However, existing studies predominantly focus on the final-step loss, leaving open whether the entire loss dynamics obey similar laws and, crucially, how the learning rate schedule (LRS) shapes them. We address these gaps in a controlled theoretical setting by analyzing stochastic gradient descent (SGD) on a power-law kernel regression model. The key insight is a novel intrinsic-time viewpoint, which captures the training progress more faithfully than iteration count. We then establish a Functional Scaling Law (FSL) that captures the full loss trajectory under arbitrary LRSs, with the schedule's influence entering through a simple convolutional functional. We further instantiate the theory for three representative LRSs -- constant, exponential decay, and warmup-stable-decay (WSD) -- and derive explicit scaling relations in both data- and compute-limited regimes. These comparisons explain key empirical phenomena: (i) higher-capacity models are more data- and compute-efficient; (ii) learning-rate decay improves training efficiency; and (iii) WSD-type schedules outperform pure decay. Finally, experiments on LLMs ranging from 0.1B to 1B parameters demonstrate the practical relevance of FSL as a surrogate model for fitting and predicting loss trajectories in large-scale pre-training.

翻译：缩放定律已成为理解和指导大语言模型训练的统一视角。然而，现有研究主要关注最终步损失，而整个损失动态是否遵循类似定律，以及学习率调度如何塑造这些动态，仍是未解之谜。我们在一个受控的理论环境中，通过分析幂律核回归模型上的随机梯度下降来填补这些空白。关键见解是一种新颖的“内在时间”视角，它比迭代次数更忠实地捕捉训练进度。随后，我们建立了一个函数缩放定律，该定律捕捉了任意学习率调度下的完整损失轨迹，其中调度的影响通过一个简单的卷积泛函引入。我们进一步将该理论实例化于三种代表性学习率调度——恒定、指数衰减和预热-稳定-衰减——并在数据受限和计算受限两种机制中推导出显式的缩放关系。这些比较解释了关键的经验现象：(i) 高容量模型具有更高的数据和计算效率；(ii) 学习率衰减提高了训练效率；(iii) 预热-稳定-衰减类调度优于纯衰减调度。最后，在参数量从0.1B到1B的大语言模型上进行的实验，证明了函数缩放定律作为拟合和预测大规模预训练中损失轨迹的代理模型具有实际相关性。