Federated Learning (FL) addresses the need to create models based on proprietary data in such a way that multiple clients retain exclusive control over their data, while all benefit from improved model accuracy due to pooled resources. Recently proposed Neural Graphical Models (NGMs) are Probabilistic Graphical models that utilize the expressive power of neural networks to learn complex non-linear dependencies between the input features. They learn to capture the underlying data distribution and have efficient algorithms for inference and sampling. We develop a FL framework which maintains a global NGM model that learns the averaged information from the local NGM models while keeping the training data within the client's environment. Our design, FedNGMs, avoids the pitfalls and shortcomings of neuron matching frameworks like Federated Matched Averaging that suffers from model parameter explosion. Our global model size remains constant throughout the process. In the cases where clients have local variables that are not part of the combined global distribution, we propose a `Stitching' algorithm, which personalizes the global NGM models by merging the additional variables using the client's data. FedNGM is robust to data heterogeneity, large number of participants, and limited communication bandwidth.

翻译：联邦学习通过聚合多方资源提升模型精度，同时确保各客户端对其专有数据享有专属控制权，以满足基于私有数据构建模型的需求。最新提出的神经图模型作为一种概率图模型，利用神经网络的表达能力学习输入特征间复杂的非线性依赖关系，能够捕捉底层数据分布并具备高效的推断与采样算法。本文设计了一种联邦学习框架，该框架维护一个全局神经图模型，在保留训练数据于客户端环境的前提下学习各局部神经图模型的平均信息。该框架（FedNGMs）避免了诸如联邦匹配平均法等神经元匹配框架因模型参数爆炸而产生的缺陷，全局模型规模在训练过程中保持恒定。针对客户端存在未纳入全局联合分布的局部变量这一情形，我们提出了"缝合"算法，通过利用客户端数据融合额外变量实现全局神经图模型的个性化。FedNGM对数据异构性、大量参与者及有限通信带宽均具有鲁棒性。