Federated bilevel optimization (FBO) has shown great potential recently in machine learning and edge computing due to the emerging nested optimization structure in meta-learning, fine-tuning, hyperparameter tuning, etc. However, existing FBO algorithms often involve complicated computations and require multiple sub-loops per iteration, each of which contains a number of communication rounds. In this paper, we propose a simple and flexible FBO framework named SimFBO, which is easy to implement without sub-loops, and includes a generalized server-side aggregation and update for improving communication efficiency. We further propose System-level heterogeneity robust FBO (ShroFBO) as a variant of SimFBO with stronger resilience to heterogeneous local computation. We show that SimFBO and ShroFBO provably achieve a linear convergence speedup with partial client participation and client sampling without replacement, as well as improved sample and communication complexities. Experiments demonstrate the effectiveness of the proposed methods over existing FBO algorithms.

翻译：去中心化双层优化（FBO）因元学习、微调、超参数调优等领域中新兴的嵌套优化结构，近年来在机器学习和边缘计算中展现出巨大潜力。然而，现有FBO算法通常涉及复杂计算，且每次迭代需要多个子循环，每个子循环均包含若干轮通信。本文提出一种名为SimFBO的简单灵活的去中心化双层优化框架，该框架无需子循环即可实现，且包含通用的服务器端聚合与更新机制以提升通信效率。我们进一步提出系统级异构鲁棒FBO（ShroFBO）作为SimFBO的变体，对异构本地计算具有更强的鲁棒性。我们证明，在部分客户端参与和不放回客户端采样条件下，SimFBO与ShroFBO可实现线性收敛加速，并同时改进样本与通信复杂度。实验表明，所提方法相较现有FBO算法具有更优性能。