This paper presents a new algorithm member for accelerating first-order methods for bilevel optimization, namely the \emph{(Perturbed) Restarted Accelerated Fully First-order methods for Bilevel Approximation}, abbreviated as \texttt{(P)RAF${}^2$BA}. The algorithm leverages \emph{fully} first-order oracles and seeks approximate stationary points in nonconvex-strongly-convex bilevel optimization, enhancing oracle complexity for efficient optimization. Theoretical guarantees for finding approximate first-order stationary points and second-order stationary points at the state-of-the-art query complexities are established, showcasing their effectiveness in solving complex optimization tasks. Empirical studies for real-world problems are provided to further validate the outperformance of our proposed algorithms. The significance of \texttt{(P)RAF${}^2$BA} in optimizing nonconvex-strongly-convex bilevel optimization problems is underscored by its state-of-the-art convergence rates and computational efficiency.

翻译：本文提出了一类用于加速双层优化的新算法成员，即用于双层近似的（扰动）重启加速完全一阶方法，简称为(P)RAF²BA。该算法利用完全一阶预言，在非凸-强凸双层优化中寻求近似驻点，提升了高效优化的预言复杂度。本文建立了在最优查询复杂度下寻找近似一阶驻点和二阶驻点的理论保证，展示了其在解决复杂优化任务中的有效性。通过面向实际问题的实证研究进一步验证了所提算法的优越性能。(P)RAF²BA在优化非凸-强凸双层优化问题中的意义，体现在其最先进的收敛速率和计算效率上。