Learning to Optimize (L2O), a technique that utilizes machine learning to learn an optimization algorithm automatically from data, has gained arising attention in recent years. A generic L2O approach parameterizes the iterative update rule and learns the update direction as a black-box network. While the generic approach is widely applicable, the learned model can overfit and may not generalize well to out-of-distribution test sets. In this paper, we derive the basic mathematical conditions that successful update rules commonly satisfy. Consequently, we propose a novel L2O model with a mathematics-inspired structure that is broadly applicable and generalized well to out-of-distribution problems. Numerical simulations validate our theoretical findings and demonstrate the superior empirical performance of the proposed L2O model.

翻译：学习优化（L2O）是一种利用机器学习自动从数据中学习优化算法的技术，近年来获得广泛关注。通用L2O方法将迭代更新规则参数化，并将更新方向视为黑箱网络进行学习。虽然通用方法具有广泛适用性，但学习到的模型可能过拟合，且难以泛化到分布外测试集。本文推导了成功更新规则通常应满足的基本数学条件。基于此，我们提出了一种具有数学启发式结构的新型L2O模型，该模型具有广泛适用性，并能良好地泛化到分布外问题。数值仿真验证了我们的理论发现，并展示了所提L2O模型的优越实证性能。