We introduce AdaSub, a stochastic optimization algorithm that computes a search direction based on second-order information in a low-dimensional subspace that is defined adaptively based on available current and past information. Compared to first-order methods, second-order methods exhibit better convergence characteristics, but the need to compute the Hessian matrix at each iteration results in excessive computational expenses, making them impractical. To address this issue, our approach enables the management of computational expenses and algorithm efficiency by enabling the selection of the subspace dimension for the search. Our code is freely available on GitHub, and our preliminary numerical results demonstrate that AdaSub surpasses popular stochastic optimizers in terms of time and number of iterations required to reach a given accuracy.

翻译：我们提出AdaSub算法，这是一种随机优化算法，它在自适应基于当前和过去可用信息定义的低维子空间中，利用二阶信息计算搜索方向。与一阶方法相比，二阶方法展现出更好的收敛特性，但每次迭代需要计算海森矩阵导致计算开销过大，使其难以实际应用。为解决该问题，我们的方法通过允许选择搜索的子空间维度，实现了计算开销与算法效率的平衡。我们的代码已在GitHub上开源，初步数值实验结果表明，在达到给定精度所需的时间和迭代次数方面，AdaSub优于流行的随机优化器。