Singular learning theory characterizes Bayesian models with non-identifiable parameterizations through two central quantities: the real log canonical threshold (RLCT), which governs marginal likelihood asymptotics, and the singular fluctuation, which determines second-order generalization behavior and the complexity term in WAIC. While the geometric meaning of the RLCT is well understood, the interpretation of singular fluctuation has remained comparatively opaque. We show that singular fluctuation admits a precise thermodynamic interpretation. Under a tempered (Gibbs) posterior, it is exactly the curvature of the Bayesian free energy with respect to inverse temperature; equivalently, the variance of the log-likelihood observable. In this sense, singular fluctuation is the statistical analogue of specific heat. This identity clarifies why singular fluctuation controls the equation of state relating training and generalization error and explains the success of WAIC in singular models: WAIC estimates a fluctuation coefficient rather than a parameter dimension. Across Gaussian mixture models and reduced-rank regression, we demonstrate that singular fluctuation behaves as a thermodynamic response coefficient. As temperature decreases, posterior reorganization suppresses fluctuation directions that affect predictive performance, and model-specific geometric observables track the decay of singular fluctuation. Rather than introducing new asymptotic expansions, this work unifies existing variance identities, equation-of-state results, and WAIC complexity corrections under a single free-energy curvature framework.
翻译:奇异学习理论通过两个核心量刻画具有不可辨识参数化的贝叶斯模型:实对数规范阈值(RLCT),它控制边际似然的渐近行为;以及奇异涨落,它决定二阶泛化行为及WAIC中的复杂度项。尽管RLCT的几何意义已被充分理解,但奇异涨落的解释仍相对模糊。本文表明,奇异涨落具有精确的热力学解释。在温度调节(吉布斯)后验下,它恰好是贝叶斯自由能关于逆温度的曲率;等价地,它是对数似然可观测量的方差。在此意义上,奇异涨落是比热的统计对应物。这一恒等式阐明了为何奇异涨落控制训练误差与泛化误差之间的状态方程,并解释了WAIC在奇异模型中的成功:WAIC估计的是涨落系数而非参数维度。通过高斯混合模型与降秩回归,我们证明奇异涨落表现为热力学响应系数。随着温度降低,后验重组抑制了影响预测性能的涨落方向,而模型特定的几何可观测量追踪奇异涨落的衰减。本文并非引入新的渐近展开,而是将现有的方差恒等式、状态方程结果和WAIC复杂度修正统一在一个自由能曲率框架下。