We develop a class of interacting particle systems for implementing a maximum marginal likelihood estimation (MMLE) procedure to estimate the parameters of a latent variable model. We achieve this by formulating a continuous-time interacting particle system which can be seen as a Langevin diffusion over an extended state space of parameters and latent variables. In particular, we prove that the parameter marginal of the stationary measure of this diffusion has the form of a Gibbs measure where number of particles acts as the inverse temperature parameter in classical settings for global optimisation. Using a particular rescaling, we then prove geometric ergodicity of this system and bound the discretisation error in a manner that is uniform in time and does not increase with the number of particles. The discretisation results in an algorithm, termed Interacting Particle Langevin Algorithm (IPLA) which can be used for MMLE. We further prove nonasymptotic bounds for the optimisation error of our estimator in terms of key parameters of the problem, and also extend this result to the case of stochastic gradients covering practical scenarios. We provide numerical experiments to illustrate the empirical behaviour of our algorithm in the context of logistic regression with verifiable assumptions. Our setting provides a straightforward way to implement a diffusion-based optimisation routine compared to more classical approaches such as the Expectation Maximisation (EM) algorithm, and allows for especially explicit nonasymptotic bounds.

翻译：我们开发了一类交互粒子系统，用于实现最大边际似然估计（MMLE）程序，以估计潜变量模型的参数。通过构建一个连续时间交互粒子系统——该可被视为在参数和潜变量扩展状态空间上的朗之万扩散过程——我们达成了这一目标。具体而言，我们证明了该扩散过程平稳测度的参数边际分布具有吉布斯测度形式，其中粒子数量充当经典全局优化设定中的逆温度参数。通过特定的尺度变换，我们进一步证明了该系统的几何遍历性，并以与时间均匀且不随粒子数量增加的方式对离散化误差进行了界定。离散化形成了一种名为交互粒子朗之万算法（IPLA）的算法，可用于MMLE。我们还根据问题的关键参数，为估计量的优化误差提供了非渐近界，并将该结果扩展到涵盖实际场景的随机梯度情形。通过数值实验，我们在满足可验证假设的逻辑回归背景下展示了算法的经验表现。与期望最大化（EM）算法等经典方法相比，我们的设定为基于扩散的优化程序提供了更直接的实现方式，并允许获得特别显式的非渐近界。