Recently, Ye et al. (Mathematical Programming 2023) designed an algorithm for solving a specific class of bilevel programs with an emphasis on applications related to hyperparameter selection, utilizing the difference of convex algorithm based on the value function approach reformulation. The proposed algorithm is particularly powerful when the lower level problem is fully convex , such as a support vector machine model or a least absolute shrinkage and selection operator model. In this paper, to suit more applications related to machine learning and statistics, we substantially weaken the underlying assumption from lower level full convexity to weak convexity. Accordingly, we propose a new reformulation using Moreau envelope of the lower level problem and demonstrate that this reformulation is a difference of weakly convex program. Subsequently, we develop a sequentially convergent algorithm for solving this difference of weakly convex program. To evaluate the effectiveness of our approach, we conduct numerical experiments on the bilevel hyperparameter selection problem from elastic net, sparse group lasso, and RBF kernel support vector machine models.

翻译：最近，Ye等人（《数学规划》2023）针对一类特定双层规划问题设计了求解算法，重点关注与超参数选择相关的应用场景，采用基于值函数方法重构的凸差算法。当下层问题完全凸时（如支持向量机模型或最小绝对收缩选择算子模型），该算法尤为有效。为适配更多机器学习和统计学应用场景，本文将对下层问题的假设从完全凸性显著弱化为弱凸性。据此，我们利用下层问题的Moreau包络提出新重构方法，并证明该重构属于弱凸差规划。随后，我们开发了用于求解此类弱凸差规划的序列收敛算法。为评估方法有效性，我们在弹性网络、稀疏组套索及径向基核支持向量机模型的双层超参数选择问题上进行了数值实验。