We propose an algorithm to solve a class of bi-level optimization problems using only first-order information. In particular, we focus on a class where the inner minimization has unique solutions. Unlike contemporary algorithms, our algorithm does not require the use of an oracle estimator for the gradient of the bi-level objective or an approximate solver for the inner problem. Instead, we alternate between descending on the inner problem using na\"ive optimization methods and descending on the upper-level objective function using specially constructed gradient estimators. We provide non-asymptotic convergence rates to stationary points of the bi-level objective in the absence of convexity of the closed-loop function and further show asymptotic convergence to only local minima of the bi-level problem. The approach is inspired by ideas from the literature on two-timescale stochastic approximation algorithms.

翻译：本文提出一种仅需一阶信息即可求解一类双层优化问题的算法。特别地，我们关注内层最小化问题具有唯一解的一类双层优化问题。与现有算法不同，本方法无需使用双层目标梯度的预言估计器或内层问题的近似求解器。我们通过交替执行两种操作来求解问题：一方面使用朴素优化方法对内层问题进行下降求解，另一方面利用特殊构造的梯度估计器对外层目标函数进行下降求解。在闭环函数非凸的情况下，我们给出了双层问题驻点的非渐近收敛率，并进一步证明了问题仅收敛至双层问题的局部极小值的渐近性。该方法受双时间尺度随机近似算法文献思想的启发。