We propose a perspective in which learning is an intrinsically dissipative process. Forgetting and regularization are not heuristic add-ons but structural requirements for adaptive systems. Drawing on information theory, thermodynamics, and information geometry, we introduce the BEDS (Bayesian Emergent Dissipative Structures) framework, modeling learning as the evolution of compressed belief states under dissipation constraints. A central contribution is the Conditional Optimality Theorem, showing that Fisher-Rao regularization measuring change via information divergence rather than Euclidean distance is the unique thermodynamically optimal regularization strategy, achieving minimal dissipation. Euclidean regularization is shown to be structurally suboptimal. The framework unifies existing methods (Ridge, SIGReg, EMA, SAC) as special cases of a single governing equation. Within this view, overfitting corresponds to over-crystallization, while catastrophic forgetting reflects insufficient dissipation control. The framework distinguishes BEDS-crystallizable problems, where beliefs converge to stable equilibria, from BEDS-maintainable problems, which require continual adaptation. It extends naturally to continual and multi-agent systems, where viability, stability under adaptation and finite resources replaces asymptotic optimality as the primary criterion. Overall, this work reframes learning as maintaining viable belief states under dissipation constraints, providing a principled lens on forgetting, regularization, and stability.
翻译:我们提出一种观点,认为学习本质上是一个耗散过程。遗忘与正则化并非启发式的附加组件,而是自适应系统的结构性要求。借鉴信息论、热力学与信息几何,我们引入BEDS(贝叶斯涌现耗散结构)框架,将学习建模为在耗散约束下压缩信念状态的演化过程。一个核心贡献是条件最优性定理,该定理表明:通过信息散度而非欧氏距离度量变化的Fisher-Rao正则化,是唯一在热力学意义上最优的正则化策略,可实现最小耗散。欧氏正则化被证明在结构上是次优的。该框架将现有方法(Ridge、SIGReg、EMA、SAC)统一为单一控制方程的特例。在此视角下,过拟合对应于过度结晶化,而灾难性遗忘则反映了耗散控制不足。该框架区分了BEDS可结晶问题(信念收敛至稳定平衡态)与BEDS需维持问题(需要持续适应)。该框架可自然扩展到持续学习与多智能体系统,其中可行性——即适应性与有限资源下的稳定性——取代渐近最优性成为主要准则。总体而言,本研究将学习重新定义为在耗散约束下维持可行信念状态的过程,为理解遗忘、正则化与稳定性提供了原理性视角。