In this paper, we propose the Bi-Sub-Gradient (Bi-SG) method, which is a generalization of the classical sub-gradient method to the setting of convex bi-level optimization problems. This is a first-order method that is very easy to implement in the sense that it requires only a computation of the associated proximal mapping or a sub-gradient of the outer non-smooth objective function, in addition to a proximal gradient step on the inner optimization problem. We show, under very mild assumptions, that Bi-SG tackles bi-level optimization problems and achieves sub-linear rates both in terms of the inner and outer objective functions. Moreover, if the outer objective function is additionally strongly convex (still could be non-smooth), the outer rate can be improved to a linear rate. Last, we prove that the distance of the generated sequence to the set of optimal solutions of the bi-level problem converges to zero.

翻译：本文提出双次梯度（Bi-SG）方法，这是经典次梯度方法在凸双层优化问题中的推广。该一阶方法实现简便，仅需计算相关近端映射或外层非光滑目标函数的次梯度，并结合内层优化问题的近端梯度步。在非常温和的假设下，我们证明Bi-SG方法能处理双层优化问题，并在内层和外层目标函数上均实现次线性收敛速率。此外，若外层目标函数额外具有强凸性（仍可保持非光滑），外层收敛速率可提升至线性。最后，我们证明所生成序列到双层问题最优解集的距离收敛于零。