Tuning effective step sizes is crucial for the stability and efficiency of optimization algorithms. While adaptive coordinate-wise step sizes tuning methods have been explored in first-order methods, second-order methods still lack efficient techniques. Current approaches, including hypergradient descent and cutting plane methods, offer limited improvements or encounter difficulties in second-order contexts. To address these challenges, we introduce a novel Learning-to-Optimize (L2O) model within the Broyden-Fletcher-Goldfarb-Shanno (BFGS) framework, which leverages neural networks to predict optimal coordinate-wise step sizes. Our model integrates a theoretical foundation that establishes conditions for the stability and convergence of these step sizes. Extensive experiments demonstrate that our approach achieves substantial improvements over traditional backtracking line search and hypergradient descent-based methods, offering up to 7$\times$ faster and stable performance across diverse optimization tasks.

翻译：调整有效的步长对于优化算法的稳定性和效率至关重要。虽然一阶方法中已经探索了自适应坐标步长调整方法，但二阶方法仍然缺乏高效的技术。当前方法，包括超梯度下降和割平面法，在二阶环境中改进有限或遇到困难。为解决这些挑战，我们在Broyden-Fletcher-Goldfarb-Shanno（BFGS）框架中引入了一种新颖的学习优化（L2O）模型，该模型利用神经网络预测最优的坐标步长。我们的模型整合了理论基础，为这些步长的稳定性和收敛性建立了条件。大量实验表明，我们的方法相比传统的回溯线搜索和基于超梯度下降的方法取得了显著改进，在多种优化任务中实现了高达7倍的加速和稳定性能。