Algorithms for bilevel optimization often encounter Hessian computations, which are prohibitive in high dimensions. While recent works offer first-order methods for unconstrained bilevel problems, the constrained setting remains relatively underexplored. We present first-order linearly constrained optimization methods with finite-time hypergradient stationarity guarantees. For linear equality constraints, we attain $\epsilon$-stationarity in $\widetilde{O}(\epsilon^{-2})$ gradient oracle calls, which is nearly-optimal. For linear inequality constraints, we attain $(\delta,\epsilon)$-Goldstein stationarity in $\widetilde{O}(d{\delta^{-1} \epsilon^{-3}})$ gradient oracle calls, where $d$ is the upper-level dimension. Finally, we obtain for the linear inequality setting dimension-free rates of $\widetilde{O}({\delta^{-1} \epsilon^{-4}})$ oracle complexity under the additional assumption of oracle access to the optimal dual variable. Along the way, we develop new nonsmooth nonconvex optimization methods with inexact oracles. We verify these guarantees with preliminary numerical experiments.

翻译：双层优化算法常涉及海森矩阵计算，这在处理高维问题时计算代价过高。尽管近期研究提出了针对无约束双层问题的一阶方法，但约束场景下的研究仍相对不足。本文提出具有有限时间超梯度平稳性保证的一阶线性约束优化方法。对于线性等式约束，我们通过 $\widetilde{O}(\epsilon^{-2})$ 次梯度预言机调用达到 $\epsilon$-平稳性，该结果近乎最优。对于线性不等式约束，我们通过 $\widetilde{O}(d{\delta^{-1} \epsilon^{-3}})$ 次梯度预言机调用达到 $(\delta,\epsilon)$-Goldstein 平稳性，其中 $d$ 表示上层问题维度。最后，在额外假设可获得最优对偶变量的预言机访问权限时，我们在线性不等式约束下获得了 $\widetilde{O}({\delta^{-1} \epsilon^{-4}})$ 的与维度无关的预言机复杂度。在研究过程中，我们开发了适用于非精确预言机的新型非光滑非凸优化方法。我们通过初步数值实验验证了这些理论保证。