For predictive modeling relying on Bayesian inversion, fully independent, or ``mean-field'', Gaussian distributions are often used as approximate probability density functions in variational inference since the number of variational parameters is twice the number of unknown model parameters. The resulting diagonal covariance structure coupled with unimodal behavior can be too restrictive when dealing with highly non-Gaussian behavior, including multimodality. High-fidelity surrogate posteriors in the form of Gaussian mixtures can capture any distribution to an arbitrary degree of accuracy while maintaining some analytical tractability. Variational inference with Gaussian mixtures with full-covariance structures suffers from a quadratic growth in variational parameters with the number of model parameters. Coupled with the existence of multiple local minima due to nonconvex trends in the loss functions often associated with variational inference, these challenges motivate the need for robust initialization procedures to improve the performance and scalability of variational inference with mixture models. In this work, we propose a method for constructing an initial Gaussian mixture model approximation that can be used to warm-start the iterative solvers for variational inference. The procedure begins with an optimization stage in model parameter space in which local gradient-based optimization, globalized through multistart, is used to determine a set of local maxima, which we take to approximate the mixture component centers. Around each mode, a local Gaussian approximation is constructed via the Laplace method. Finally, the mixture weights are determined through constrained least squares regression. Robustness and scalability are demonstrated using synthetic tests. The methodology is applied to an inversion problem in structural dynamics involving unknown viscous damping coefficients.

翻译：在依赖贝叶斯反演的预测建模中，由于变分参数数量是未知模型参数数量的两倍，完全独立（即“平均场”）的高斯分布常被用作变分推断中的近似概率密度函数。然而，这种对角协方差结构与单模态特性在处理包含多模态等强非高斯行为时可能过于严格。以高斯混合形式的高保真替代后验分布能够在保持一定解析可追踪性的同时，以任意精度逼近任何分布。采用全协方差结构的高斯混合进行变分推断时，变分参数会随模型参数数量呈二次增长。加之变分推断中损失函数非凸性导致的多个局部极小值问题，这些挑战催生了鲁棒初始化方法的需求，以提升混合模型变分推断的性能与可扩展性。本文提出一种构建初始高斯混合模型近似的方法，为变分推断的迭代求解器提供热启动。该方法首先在模型参数空间进行优化阶段：通过多起点策略全局化的局部梯度优化确定一组局部极大值，将其近似为混合分量中心；随后围绕每个模态利用拉普拉斯方法构建局部高斯近似；最终通过约束最小二乘回归确定混合权重。合成测试验证了该方法的鲁棒性与可扩展性，并将其应用于涉及未知粘性阻尼系数的结构动力学反演问题。