Federated bilevel optimization (FBO) has shown great potential recently in machine learning and edge computing due to the emerging nested optimization structure in meta-learning, fine-tuning, hyperparameter tuning, etc. However, existing FBO algorithms often involve complicated computations and require multiple sub-loops per iteration, each of which contains a number of communication rounds. In this paper, we propose a simple and flexible FBO framework named SimFBO, which is easy to implement without sub-loops, and includes a generalized server-side aggregation and update for improving communication efficiency. We further propose System-level heterogeneity robust FBO (ShroFBO) as a variant of SimFBO with stronger resilience to heterogeneous local computation. We show that SimFBO and ShroFBO provably achieve a linear convergence speedup with partial client participation and client sampling without replacement, as well as improved sample and communication complexities. Experiments demonstrate the effectiveness of the proposed methods over existing FBO algorithms.

翻译：联邦双层优化（FBO）因元学习、微调、超参数调优等新兴嵌套优化结构，近期在机器学习与边缘计算领域展现出巨大潜力。然而，现有FBO算法往往涉及复杂计算，且每次迭代需执行多个子循环，每个子循环包含多轮通信。本文提出名为SimFBO的简单灵活联邦双层优化框架，该框架无需子循环即可轻松实现，并包含通用化服务器端聚合与更新机制以提升通信效率。我们进一步提出系统级异构鲁棒FBO（ShroFBO）作为SimFBO的变体，其对异构本地计算具有更强的鲁棒性。理论证明，在部分客户端参与且无放回客户端采样条件下，SimFBO与ShroFBO能够实现线性收敛加速，同时获得更优的样本与通信复杂度。实验结果表明，所提方法相较现有FBO算法具有显著有效性。