We have formulated a family of machine learning problems as the time evolution of Parametric Probabilistic Models (PPMs), inherently rendering a thermodynamic process. Our primary motivation is to leverage the rich toolbox of thermodynamics of information to assess the information-theoretic content of learning a probabilistic model. We first introduce two information-theoretic metrics: Memorized-information (M-info) and Learned-information (L-info), which trace the flow of information during the learning process of PPMs. Then, we demonstrate that the accumulation of L-info during the learning process is associated with entropy production, and parameters serve as a heat reservoir in this process, capturing learned information in the form of M-info.

翻译：我们将一类机器学习问题形式化为参数化概率模型（PPMs）的时间演化过程，本质上将其视为一种热力学过程。主要动机在于利用信息热力学的丰富工具来评估学习概率模型的信息论内容。首先，我们引入两种信息论度量：记忆信息（M-info）和学习信息（L-info），用于追踪PPMs学习过程中信息流的变化。随后，我们证明学习过程中L-info的累积与熵产生相关联，其中参数在此过程中扮演热库角色，以M-info的形式捕获已学习信息。