Federated bilevel optimization (FBO) has shown great potential recently in machine learning and edge computing due to the emerging nested optimization structure in meta-learning, fine-tuning, hyperparameter tuning, etc. However, existing FBO algorithms often involve complicated computations and require multiple sub-loops per iteration, each of which contains a number of communication rounds. In this paper, we propose a simple and flexible FBO framework named SimFBO, which is easy to implement without sub-loops, and includes a generalized server-side aggregation and update for improving communication efficiency. We further propose System-level heterogeneity robust FBO (ShroFBO) as a variant of SimFBO with stronger resilience to heterogeneous local computation. We show that SimFBO and ShroFBO provably achieve a linear convergence speedup with partial client participation and client sampling without replacement, as well as improved sample and communication complexities. Experiments demonstrate the effectiveness of the proposed methods over existing FBO algorithms.

翻译：[translated abstract in Chinese] 联邦双层优化（FBO）近年来在机器学习和边缘计算中展现出巨大潜力，这得益于元学习、微调、超参数调优等场景中出现的嵌套优化结构。然而，现有的FBO算法通常涉及复杂计算，且每次迭代需要多个子循环，每个子循环包含多轮通信。本文提出一个简单灵活的FBO框架SimFBO，其无需子循环即可轻松实现，并通过广义服务器端聚合与更新机制提升通信效率。我们进一步提出系统级异构鲁棒FBO（ShroFBO）作为SimFBO的变体，对异构本地计算具有更强的鲁棒性。本文证明，在部分客户端参与和无放回客户端采样条件下，SimFBO和ShroFBO可收敛至线性加速比，并实现更优的样本与通信复杂度。实验表明，所提方法在现有FBO算法中具有显著有效性。