Stochastic approximation with multiple coupled sequences (MSA) has found broad applications in machine learning as it encompasses a rich class of problems including bilevel optimization (BLO), multi-level compositional optimization (MCO), and reinforcement learning (specifically, actor-critic methods). However, designing provably-efficient federated algorithms for MSA has been an elusive question even for the special case of double sequence approximation (DSA). Towards this goal, we develop FedMSA which is the first federated algorithm for MSA, and establish its near-optimal communication complexity. As core novelties, (i) FedMSA enables the provable estimation of hypergradients in BLO and MCO via local client updates, which has been a notable bottleneck in prior theory, and (ii) our convergence guarantees are sensitive to the heterogeneity-level of the problem. We also incorporate momentum and variance reduction techniques to achieve further acceleration leading to near-optimal rates. Finally, we provide experiments that support our theory and demonstrate the empirical benefits of FedMSA. As an example, FedMSA enables order-of-magnitude savings in communication rounds compared to prior federated BLO schemes.

翻译：多耦合序列随机逼近（MSA）在机器学习领域具有广泛应用，因为它涵盖了一类丰富的问题，包括双层优化（BLO）、多级组合优化（MCO）和强化学习（特别是actor-critic方法）。然而，设计可证明高效的MSA联邦算法一直是一个棘手的问题，即使是双序列逼近（DSA）这一特例也不例外。为此，我们提出了FedMSA——首个针对MSA问题的联邦算法，并建立了其近乎最优的通信复杂度。核心创新点在于：（i）FedMSA通过局部客户端更新实现了BLO和MCO中超梯度的可证明估计，这在此前理论中是一个显著的瓶颈；（ii）我们的收敛保证对问题的异质性水平具有敏感性。我们还采用了动量与方差缩减技术以实现进一步加速，从而获得近乎最优的收敛速率。最后，我们通过实验验证了理论分析，并展示了FedMSA的实际优势。例如，与先前的联邦BLO方案相比，FedMSA在通信轮次上实现了数量级的节省。