This paper presents a new algorithm member for accelerating first-order methods for bilevel optimization, namely the \emph{(Perturbed) Restarted Accelerated Fully First-order methods for Bilevel Approximation}, abbreviated as \texttt{(P)RAF${}^2$BA}. The algorithm leverages \emph{fully} first-order oracles and seeks approximate stationary points in nonconvex-strongly-convex bilevel optimization, enhancing oracle complexity for efficient optimization. Theoretical guarantees for finding approximate first-order stationary points and second-order stationary points at the state-of-the-art query complexities are established, showcasing their effectiveness in solving complex optimization tasks. Empirical studies for real-world problems are provided to further validate the outperformance of our proposed algorithms. The significance of \texttt{(P)RAF${}^2$BA} in optimizing nonconvex-strongly-convex bilevel optimization problems is underscored by its state-of-the-art convergence rates and computational efficiency.

翻译：本文提出一种加速双层优化一阶方法的新算法成员，即面向双层逼近的（扰动）重启加速完全一阶方法，缩写为\texttt{(P)RAF${}^2$BA}。该算法利用\emph{完全}一阶预言，在非凸-强凸双层优化中寻求近似驻点，从而提升优化效率的预言复杂度。本文建立了以最先进的查询复杂度寻找近似一阶驻点与二阶驻点的理论保证，展示了其在解决复杂优化任务中的有效性。针对现实问题的实证研究进一步验证了所提算法的优越性能。\texttt{(P)RAF${}^2$BA}凭借其最先进的收敛速率与计算效率，在优化非凸-强凸双层优化问题中的重要性得到充分彰显。