Gradient methods are experiencing a growth in methodological and theoretical developments owing to the challenges posed by optimization problems arising in data science. However, such gradient methods face diverging optimality gaps or exploding objective evaluations when applied to optimization problems with realistic properties for data science applications. In this work, we address this gap by developing a generic methodology that economically uses objective function evaluations in a problem-driven manner to prevent optimality gap divergence and avoid explosions in objective evaluations. Our methodology allows for a variety of step size routines and search direction strategies. Furthermore, we develop a particular, novel step size selection methodology that is well-suited to our framework. We show that our specific procedure is highly competitive with standard optimization methods on CUTEst test problems. We then show our specific procedure is highly favorable relative to standard optimization methods on a particularly tough data science problem: learning the parameters in a generalized estimating equation model. Thus, we provide a novel gradient methodology that is better suited to optimization problems from this important class of data science applications.

翻译：梯度方法因数据科学中优化问题带来的挑战而在方法论和理论发展上不断增长。然而，当应用于具有数据科学应用现实特性的优化问题时，这些梯度方法面临最优性差距发散或目标函数评估爆炸的问题。本研究通过开发一种通用方法论来填补这一空白，该方法论以问题驱动的方式经济地使用目标函数评估，从而防止最优性差距发散并避免目标函数评估爆炸。我们的方法论支持多种步长策略和搜索方向策略。此外，我们提出了一种特定且新颖的步长选择方法，该方法与我们的框架高度契合。实验表明，我们的特定程序在CUTEst测试问题上与标准优化方法相比具有高度竞争力。进一步地，在一项特别具有挑战性的数据科学问题——广义估计方程模型参数学习中，我们的特定程序相对于标准优化方法展现出显著优势。因此，我们提供了一种新颖的梯度方法论，能够更好地适用于这一重要数据科学应用类别的优化问题。