Efficient inference in high-dimensional models remains a central challenge in machine learning. This paper introduces the Gaussian Ensemble Belief Propagation (GEnBP) algorithm, a fusion of the Ensemble Kalman filter and Gaussian Belief Propagation (GaBP) methods. GEnBP updates ensembles by passing low-rank local messages over a graphical model. This combination inherits favourable qualities from each method. Ensemble techniques allow GEnBP to handle high-dimensional states, parameters and intricate, noisy, black-box generation processes. The use of local messages in a graphical model structure ensures that the approach can efficiently handle complex dependence structures. GEnBP is advantageous when the ensemble size may be considerably smaller than the inference dimension. This scenario often arises in fields such as spatiotemporal modelling, image processing and physical model inversion. GEnBP can be applied to general problem structures, including data assimilation, system identification and hierarchical models. Supporting code is available at https://github.com/danmackinlay/GEnBP
翻译:高效推断高维模型仍然是机器学习领域的核心挑战。本文提出了高斯集成置信传播算法,该方法融合了集成卡尔曼滤波与高斯置信传播技术。GEnBP通过在概率图模型上传递低秩局部信息来更新集成样本。这种融合继承了两种方法各自的优势:集成技术使GEnBP能够处理高维状态变量、参数以及复杂含噪声的黑箱生成过程;而概率图模型中的局部信息传递机制确保该方法能高效处理复杂的依赖结构。当集成规模远小于推断维度时,GEnBP展现出独特优势,这种场景常见于时空建模、图像处理和物理模型反演等领域。该算法可应用于包括数据同化、系统辨识和层次模型在内的通用问题结构。相关代码已发布于https://github.com/danmackinlay/GEnBP。