We systematically study various network Expectation-Maximization (EM) algorithms for the Gaussian mixture model within the framework of decentralized federated learning. Our theoretical investigation reveals that directly extending the classical decentralized supervised learning method to the EM algorithm exhibits poor estimation accuracy with heterogeneous data across clients and struggles to converge numerically when Gaussian components are poorly-separated. To address these issues, we propose two novel solutions. First, to handle heterogeneous data, we introduce a momentum network EM (MNEM) algorithm, which uses a momentum parameter to combine information from both the current and historical estimators. Second, to tackle the challenge of poorly-separated Gaussian components, we develop a semi-supervised MNEM (semi-MNEM) algorithm, which leverages partially labeled data. Rigorous theoretical analysis demonstrates that MNEM can achieve statistical efficiency comparable to that of the whole sample estimator when the mixture components satisfy certain separation conditions, even in heterogeneous scenarios. Moreover, the semi-MNEM estimator enhances the convergence speed of the MNEM algorithm, effectively addressing the numerical convergence challenges in poorly-separated scenarios. Extensive simulation and real data analyses are conducted to justify our theoretical findings.

翻译：我们系统研究了去中心化联邦学习框架下高斯混合模型的各种网络期望最大化（EM）算法。理论分析表明，将经典的去中心化监督学习方法直接扩展至EM算法时，在客户端数据异构的情况下估计精度较差，且当高斯分量分离不佳时难以实现数值收敛。为解决这些问题，我们提出了两种创新方案。首先，针对数据异构问题，我们提出了动量网络EM（MNEM）算法，该算法通过动量参数融合当前估计量与历史估计量的信息。其次，为应对高斯分量分离不佳的挑战，我们开发了半监督MNEM（semi-MNEM）算法，该算法利用部分标注数据。严格的理论分析证明，当混合分量满足特定分离条件时，即使存在数据异构，MNEM也能达到与全样本估计量相当的统计效率。此外，半监督MNEM估计量显著提升了MNEM算法的收敛速度，有效解决了分量分离不佳场景下的数值收敛难题。我们通过大量仿真实验与真实数据分析验证了理论结论。