Federated learning methods, that is, methods that perform model training using data situated across different sources, whilst simultaneously not having the data leave their original source, are of increasing interest in a number of fields. However, despite this interest, the classes of models for which easily-applicable and sufficiently general approaches are available is limited, excluding many structured probabilistic models. We present a general yet elegant resolution to the aforementioned issue. The approach is based on adopting structured variational inference, an approach widely used in Bayesian machine learning, to the federated setting. Additionally, a communication-efficient variant analogous to the canonical FedAvg algorithm is explored. The effectiveness of the proposed algorithms are demonstrated, and their performance is compared on Bayesian multinomial regression, topic modelling, and mixed model examples.

翻译：联邦学习方法，即利用分布在多个源头的数据进行模型训练、同时确保数据不离开原始源头的方法，在多个领域中日益受到关注。然而，尽管兴趣浓厚，目前易于应用且具有充分通用性的模型类别仍然有限，排除了许多结构化概率模型。我们针对上述问题提出了一种通用而优雅的解决方案。该方法基于将结构化变分推断（一种在贝叶斯机器学习中广泛使用的技术）应用于联邦场景。此外，我们还探索了类似于经典FedAvg算法的通信高效变体。通过贝叶斯多项回归、主题建模及混合模型示例，我们展示了所提算法的有效性，并对其性能进行了比较。