Bilevel optimization has received more and more attention recently due to its wide applications in machine learning. In this paper, we consider bilevel optimization in decentralized networks. In particular, we propose a novel single-loop algorithm for solving decentralized bilevel optimization with strongly convex lower level problem. Our algorithm is fully single-loop and does not require heavy matrix-vector multiplications when approximating the hypergradient. Moreover, unlike existing methods for decentralized bilevel optimization and federated bilevel optimization, our algorithm does not require any gradient heterogeneity assumption. Our analysis shows that the proposed algorithm achieves the best known convergence rate for bilevel optimization algorithms.

翻译：双层优化因其在机器学习中的广泛应用，近年来受到越来越多的关注。本文考虑去中心化网络中的双层优化问题。具体而言，我们提出了一种新颖的单循环算法，用于求解具有强凸下层问题的去中心化双层优化。该算法完全是单循环的，且在近似超梯度时无需进行繁重的矩阵-向量乘法。此外，与现有的去中心化双层优化和联邦双层优化方法不同，我们的算法不需要任何梯度异质性假设。分析表明，所提出的算法达到了双层优化算法已知的最优收敛速率。