In this paper, we study a class of bilevel optimization problems, also known as simple bilevel optimization, where we minimize a smooth objective function over the optimal solution set of another convex constrained optimization problem. Several iterative methods have been developed for tackling this class of problems. Alas, their convergence guarantees are either asymptotic for the upper-level objective, or the convergence rates are slow and sub-optimal. To address this issue, in this paper, we introduce a novel bilevel optimization method that locally approximates the solution set of the lower-level problem via a cutting plane, and then runs a conditional gradient update to decrease the upper-level objective. When the upper-level objective is convex, we show that our method requires ${\mathcal{O}}(\max\{1/\epsilon_f,1/\epsilon_g\})$ iterations to find a solution that is $\epsilon_f$-optimal for the upper-level objective and $\epsilon_g$-optimal for the lower-level objective. Moreover, when the upper-level objective is non-convex, our method requires ${\mathcal{O}}(\max\{1/\epsilon_f^2,1/(\epsilon_f\epsilon_g)\})$ iterations to find an $(\epsilon_f,\epsilon_g)$-optimal solution. We also prove stronger convergence guarantees under the H\"olderian error bound assumption on the lower-level problem. To the best of our knowledge, our method achieves the best-known iteration complexity for the considered class of bilevel problems.

翻译：本文研究一类双层优化问题，该类问题亦称为简单双层优化，目标是在另一个凸约束优化问题的最优解集上最小化光滑目标函数。已有多种迭代方法被用于处理此类问题。然而，这些方法的收敛性保证要么是关于上层目标函数的渐近收敛，要么收敛速度缓慢且非最优。为解决这一问题，本文提出一种新型双层优化方法，该方法通过切割平面对下层问题的解集进行局部近似，随后执行条件梯度更新以降低上层目标函数值。当上层目标函数为凸时，我们证明该方法需要${\mathcal{O}}(\max\{1/\epsilon_f,1/\epsilon_g\})$次迭代即可找到满足上层目标$\epsilon_f$-最优且下层目标$\epsilon_g$-最优的解。此外，当上层目标函数非凸时，该方法需要${\mathcal{O}}(\max\{1/\epsilon_f^2,1/(\epsilon_f\epsilon_g)\})$次迭代即可找到$(\epsilon_f,\epsilon_g)$-最优解。在下层问题满足Hölderian误差界假设的条件下，我们还证明了更强的收敛性保证。据我们所知，本文方法在所述双层优化问题类别中达到了最先进的迭代复杂度。