In this paper, we provide a multiscale perspective on the problem of maximum marginal likelihood estimation. We consider and analyse a diffusion-based maximum marginal likelihood estimation scheme using ideas from multiscale dynamics. Our perspective is based on stochastic averaging; we make an explicit connection between ideas in applied probability and parameter inference in computational statistics. In particular, we consider a general class of coupled Langevin diffusions for joint inference of latent variables and parameters in statistical models, where the latent variables are sampled from a fast Langevin process (which acts as a sampler), and the parameters are updated using a slow Langevin process (which acts as an optimiser). We show that the resulting system of stochastic differential equations (SDEs) can be viewed as a two-time scale system. To demonstrate the utility of such a perspective, we show that the averaged parameter dynamics obtained in the limit of scale separation can be used to estimate the optimal parameter, within the strongly convex setting. We do this by using recent uniform-in-time non-asymptotic averaging bounds. Finally, we conclude by showing that the slow-fast algorithm we consider here, termed Slow-Fast Langevin Algorithm, performs on par with state-of-the-art methods on a variety of examples. We believe that the stochastic averaging approach we provide in this paper enables us to look at these algorithms from a fresh angle, as well as unlocking the path to develop and analyse new methods using well-established averaging principles.

翻译：本文从多尺度动力学的角度，为最大边际似然估计问题提供了一个多尺度视角。我们考虑并分析了一种基于扩散的最大边际似然估计方案，该方案利用了多尺度动力学的思想。我们的视角基于随机平均化；我们明确建立了应用概率论思想与计算统计学中参数推断之间的联系。具体而言，我们考虑了一类用于统计模型中潜变量与参数联合推断的耦合Langevin扩散，其中潜变量从一个快速的Langevin过程（作为采样器）中采样，而参数则使用一个缓慢的Langevin过程（作为优化器）进行更新。我们证明了所得到的随机微分方程组可以视为一个双时间尺度系统。为了证明这种视角的实用性，我们展示了在尺度分离极限下获得的平均参数动力学可用于估计最优参数（在强凸设定下）。我们通过使用最新的、时间一致的非渐近平均界来证明这一点。最后，我们通过展示本文所考虑的慢-快算法（称为慢-快Langevin算法）在各种示例中与最先进方法性能相当来总结全文。我们相信，本文提供的随机平均化方法使我们能够从一个全新的角度审视这些算法，并为利用成熟的平均化原理开发和分析新方法开辟了道路。