Stochastic bilevel optimization (SBO) is becoming increasingly essential in machine learning due to its versatility in handling nested structures. To address large-scale SBO, decentralized approaches have emerged as effective paradigms in which nodes communicate with immediate neighbors without a central server, thereby improving communication efficiency and enhancing algorithmic robustness. However, current decentralized SBO algorithms face challenges, including expensive inner-loop updates and unclear understanding of the influence of network topology, data heterogeneity, and the nested bilevel algorithmic structures. In this paper, we introduce a single-loop decentralized SBO (D-SOBA) algorithm and establish its transient iteration complexity, which, for the first time, clarifies the joint influence of network topology and data heterogeneity on decentralized bilevel algorithms. D-SOBA achieves the state-of-the-art asymptotic rate, asymptotic gradient/Hessian complexity, and transient iteration complexity under more relaxed assumptions compared to existing methods. Numerical experiments validate our theoretical findings.

翻译：随机双层优化因其在处理嵌套结构中的通用性，在机器学习中变得日益重要。为解决大规模双层优化问题，去中心化方法已成为有效的范式，其中节点无需中央服务器即可与相邻节点通信，从而提升通信效率并增强算法鲁棒性。然而，当前去中心化双层优化算法面临挑战，包括昂贵的内层循环更新，以及对网络拓扑、数据异质性和嵌套双层算法结构影响的认知不清晰。本文提出一种单循环去中心化双层优化算法，并建立其瞬态迭代复杂度，首次阐明了网络拓扑与数据异质性对去中心化双层算法的联合影响。与现有方法相比，该算法在更宽松的假设下实现了最先进的渐近速率、渐近梯度/海森矩阵复杂度及瞬态迭代复杂度。数值实验验证了我们的理论发现。