Standard federated optimization methods successfully apply to stochastic problems with single-level structure. However, many contemporary ML problems -- including adversarial robustness, hyperparameter tuning, and actor-critic -- fall under nested bilevel programming that subsumes minimax and compositional optimization. In this work, we propose \fedblo: A federated alternating stochastic gradient method to address general nested problems. We establish provable convergence rates for \fedblo in the presence of heterogeneous data and introduce variations for bilevel, minimax, and compositional optimization. \fedblo introduces multiple innovations including federated hypergradient computation and variance reduction to address inner-level heterogeneity. We complement our theory with experiments on hyperparameter \& hyper-representation learning and minimax optimization that demonstrate the benefits of our method in practice. Code is available at https://github.com/ucr-optml/FedNest.

翻译：标准联邦优化方法成功地适用于单层结构的随机问题。然而,许多当代 ML 问题 -- -- 包括对抗性强力、超参数调试和演员-critic -- -- 都属于嵌入双层编程的嵌入式双层编程中,这种编程中包含微缩和成份优化。在这项工作中,我们建议采用\ fedblo: 一种交替交替互换的梯度方法来解决一般的嵌入问题。我们为\ fedblo 制定了可辨别的调合率,并引入双层、微模和成份优化的变异性。\ fedblo 引入了多种创新,包括联合高梯度计算和变异性减法,以解决内层异性。我们用超双立计-超度学习和微缩增度优化实验来补充我们的理论,以展示我们方法的实际效益。代码可在 https://github.com/ucr-optml/FedNest查阅。