Federated bilevel optimization (FBO) has shown great potential recently in machine learning and edge computing due to the emerging nested optimization structure in meta-learning, fine-tuning, hyperparameter tuning, etc. However, existing FBO algorithms often involve complicated computations and require multiple sub-loops per iteration, each of which contains a number of communication rounds. In this paper, we propose a simple and flexible FBO framework named SimFBO, which is easy to implement without sub-loops, and includes a generalized server-side aggregation and update for improving communication efficiency. We further propose System-level heterogeneity robust FBO (ShroFBO) as a variant of SimFBO with stronger resilience to heterogeneous local computation. We show that SimFBO and ShroFBO provably achieve a linear convergence speedup with partial client participation and client sampling without replacement, as well as improved sample and communication complexities. Experiments demonstrate the effectiveness of the proposed methods over existing FBO algorithms.

翻译：联邦双层优化（FBO）由于在元学习、微调、超参数调优等领域中出现的嵌套优化结构，近年来在机器学习和边缘计算中展现出巨大潜力。然而，现有FBO算法通常涉及复杂计算，且每次迭代需要多个子循环，每个子循环包含多轮通信。本文提出一种简单灵活的FBO框架——SimFBO，其无需子循环即可实现，同时包含广义服务器端聚合与更新机制以提高通信效率。我们进一步提出系统异构鲁棒FBO（ShroFBO）作为SimFBO的变体，增强了对异构本地计算的鲁棒性。理论分析表明，在部分客户端参与和无放回客户端采样条件下，SimFBO与ShroFBO可证明实现线性加速收敛，并提升样本与通信复杂度。实验结果验证了所提方法相较现有FBO算法的有效性。