Bilevel optimization has gained significant attention in recent years due to its broad applications in machine learning. This paper focuses on bilevel optimization in decentralized networks and proposes a novel single-loop algorithm for solving decentralized bilevel optimization with a strongly convex lower-level problem. Our approach is a fully single-loop method that approximates the hypergradient using only two matrix-vector multiplications per iteration. Importantly, our algorithm does not require any gradient heterogeneity assumption, distinguishing it from existing methods for decentralized bilevel optimization and federated bilevel optimization. Our analysis demonstrates that the proposed algorithm achieves the best-known convergence rate for bilevel optimization algorithms. We also present experimental results on hyperparameter optimization problems using both synthetic and MNIST datasets, which demonstrate the efficiency of our proposed algorithm.

翻译：双层优化因其在机器学习中的广泛应用而近年来备受关注。本文聚焦于去中心化网络中的双层优化问题，并提出一种新颖的单循环算法，用于求解下层问题为强凸函数的去中心化双层优化。我们的方法是一种完全单循环算法，每次迭代仅需两次矩阵向量乘法即可近似计算超梯度。重要的是，我们的算法无需任何梯度异质性假设，这使其区别于现有的去中心化双层优化及联邦双层优化方法。理论分析表明，所提算法达到了双层优化算法中已知的最优收敛速率。我们还给出了基于合成数据集和MNIST数据集的超参数优化实验，实验结果证明了所提算法的效率。