In the paper, we propose an effective and efficient Compositional Federated Learning (ComFedL) algorithm for solving a new compositional Federated Learning (FL) framework, which frequently appears in many data mining and machine learning problems with a hierarchical structure such as distributionally robust FL and model-agnostic meta learning (MAML). Moreover, we study the convergence analysis of our ComFedL algorithm under some mild conditions, and prove that it achieves a convergence rate of $O(\frac{1}{\sqrt{T}})$, where $T$ denotes the number of iteration. To the best of our knowledge, our new Compositional FL framework is the first work to bridge federated learning with composition stochastic optimization. In particular, we first transform the distributionally robust FL (i.e., a minimax optimization problem) into a simple composition optimization problem by using KL divergence regularization. At the same time, we also first transform the distribution-agnostic MAML problem (i.e., a minimax optimization problem) into a simple yet effective composition optimization problem. Finally, we apply two popular machine learning tasks, i.e., distributionally robust FL and MAML to demonstrate the effectiveness of our algorithm.

翻译：在本文中，我们提出了一种高效且有效的组合式联邦学习（ComFedL）算法，用于解决一种新颖的组合式联邦学习（FL）框架。该框架常见于具有层次结构的许多数据挖掘与机器学习问题中，例如分布鲁棒FL和模型无关元学习（MAML）。此外，我们在温和条件下研究了ComFedL算法的收敛性分析，并证明其收敛速率为$O(\frac{1}{\sqrt{T}})$，其中$T$表示迭代次数。据我们所知，我们的新组合式FL框架是首个将联邦学习与组合随机优化相结合的工作。特别地，我们首先利用KL散度正则化将分布鲁棒FL（即一个极小极大优化问题）转化为一个简单的组合优化问题。同时，我们也首次将分布无关的MAML问题（即一个极小极大优化问题）转化为一个简单而有效的组合优化问题。最后，我们应用了两个流行的机器学习任务，即分布鲁棒FL和MAML，来验证我们算法的有效性。