We propose a continuous-time formulation of persistent contrastive divergence (PCD) for maximum likelihood estimation (MLE) of unnormalised densities. Our approach expresses PCD as a coupled, multiscale system of stochastic differential equations (SDEs), which perform optimisation of the parameter and sampling of the associated parametrised density, simultaneously. From this novel formulation, we are able to derive explicit bounds for the error between the PCD iterates and the MLE solution for the model parameter. This is made possible by deriving uniform-in-time (UiT) bounds for the difference in moments between the multiscale system and the averaged regime. An efficient implementation of the continuous-time scheme is introduced, leveraging a class of explicit, stable intregators, stochastic orthogonal Runge-Kutta Chebyshev (S-ROCK), for which we provide explicit error estimates in the long-time regime. This leads to a novel method for training energy-based models (EBMs) with explicit error guarantees.

翻译：我们提出了一种用于未归一化密度最大似然估计的持续对比散度连续时间公式。我们的方法将PCD表达为一个耦合的多尺度随机微分方程系统，该系统同时执行参数优化和相关参数化密度的采样。基于这一新颖的公式，我们能够推导出PCD迭代与模型参数最大似然估计解之间误差的显式界。这是通过推导多尺度系统与平均状态之间矩差异的均匀时间界实现的。我们引入了一种连续时间方案的高效实现，利用了一类显式稳定的积分器——随机正交龙格-库塔切比雪夫方法，并为其在长时间状态下的误差提供了显式估计。这为训练具有显式误差保证的能量基模型提供了一种新方法。