We develop a class of interacting particle systems for implementing a maximum marginal likelihood estimation (MMLE) procedure to estimate the parameters of a latent variable model. We achieve this by formulating a continuous-time interacting particle system which can be seen as a Langevin diffusion over an extended state space of parameters and latent variables. In particular, we prove that the parameter marginal of the stationary measure of this diffusion has the form of a Gibbs measure where number of particles acts as the inverse temperature parameter in classical settings for global optimisation. Using a particular rescaling, we then prove geometric ergodicity of this system and bound the discretisation error in a manner that is uniform in time and does not increase with the number of particles. The discretisation results in an algorithm, termed Interacting Particle Langevin Algorithm (IPLA) which can be used for MMLE. We further prove nonasymptotic bounds for the optimisation error of our estimator in terms of key parameters of the problem, and also extend this result to the case of stochastic gradients covering practical scenarios. We provide numerical experiments to illustrate the empirical behaviour of our algorithm in the context of logistic regression with verifiable assumptions. Our setting provides a straightforward way to implement a diffusion-based optimisation routine compared to more classical approaches such as the Expectation Maximisation (EM) algorithm, and allows for especially explicit nonasymptotic bounds.

翻译：我们开发了一类交互粒子系统，用于实现最大边缘似然估计（MMLE）程序以估计潜变量模型的参数。我们通过构建一个连续时间交互粒子系统来实现这一目标，该系统可被视为在参数和潜变量的扩展状态空间上的朗之万扩散。特别地，我们证明了该扩散平稳测度的参数边缘具有吉布斯测度的形式，其中粒子数量扮演了经典全局优化设置中逆温度参数的角色。通过特定的重缩放，我们进一步证明了该系统的几何遍历性，并以在时间上均匀且不随粒子数量增加的方式限制了离散化误差。该离散化过程产生了一种称为交互粒子朗之万算法（IPLA）的算法，可用于MMLE。我们进一步证明了估计器优化误差关于问题关键参数的非渐近界，并将此结果扩展到涵盖实际场景的随机梯度情形。我们提供了数值实验，以在具有可验证假设的逻辑回归背景下说明算法的经验行为。与期望最大化（EM）算法等更经典的方法相比，我们的设置提供了一种更直接的基于扩散的优化实现方式，并允许得到特别显式的非渐近界。