Efficient inference in high-dimensional models remains a central challenge in machine learning. This paper introduces the Gaussian Ensemble Belief Propagation (GEnBP) algorithm, a fusion of the Ensemble Kalman filter and Gaussian Belief Propagation (GaBP) methods. GEnBP updates ensembles by passing low-rank local messages over a graphical model. This combination inherits favourable qualities from each method. Ensemble techniques allow GEnBP to handle high-dimensional states, parameters and intricate, noisy, black-box generation processes. The use of local messages in a graphical model structure ensures that the approach can efficiently handle complex dependence structures. GEnBP is advantageous when the ensemble size may be considerably smaller than the inference dimension. This scenario often arises in fields such as spatiotemporal modelling, image processing and physical model inversion. GEnBP can be applied to general problem structures, including data assimilation, system identification and hierarchical models. Supporting code is available at https://github.com/danmackinlay/GEnBP
翻译:高维模型中的高效推理仍是机器学习领域的核心挑战。本文提出高斯集成置信传播(GEnBP)算法,该算法融合了集合卡尔曼滤波与高斯置信传播(GaBP)方法。GEnBP通过在概率图模型上传递低秩局部消息来更新集成样本,这种结合继承了两种方法的优良特性。集成技术使GEnBP能够处理高维状态、参数以及复杂嘈杂的黑盒生成过程,而利用图模型结构中的局部消息则确保该方法能高效处理复杂依赖结构。当集成规模可能远小于推理维度时(常见于时空建模、图像处理和物理模型反演等领域),GEnBP具有显著优势。该算法可适用于一般性问题结构,包括数据同化、系统辨识及分层模型。支持代码见https://github.com/danmackinlay/GEnBP