Fast gradient-based optimization algorithms have become increasingly essential for the computationally efficient training of machine learning models. One technique is to multiply the gradient by a preconditioner matrix to produce a step, but it is unclear what the best preconditioner matrix is. This paper introduces a novel machine learning optimizer called LODO, which tries to online meta-learn the best preconditioner during optimization. Specifically, our optimizer merges Learning to Optimize (L2O) techniques with quasi-Newton methods to learn preconditioners parameterized as neural networks; they are more flexible than preconditioners in other quasi-Newton methods. Unlike other L2O methods, LODO does not require any meta-training on a training task distribution, and instead learns to optimize on the fly while optimizing on the test task, adapting to the local characteristics of the loss landscape while traversing it. Theoretically, we show that our optimizer approximates the inverse Hessian in noisy loss landscapes and is capable of representing a wide range of inverse Hessians. We experimentally verify that our algorithm can optimize in noisy settings, and show that simpler alternatives for representing the inverse Hessians worsen performance. Lastly, we use our optimizer to train a semi-realistic deep neural network with 95k parameters at speeds comparable to those of standard neural network optimizers.

翻译：基于梯度的快速优化算法对于机器学习模型的高效训练日益重要。一种技术是将梯度乘以预条件矩阵以生成步长，但最优的预条件矩阵尚不明确。本文提出了一种名为LODO的新型机器学习优化器，它试图在优化过程中在线元学习最佳预条件矩阵。具体来说，我们的优化器将学习优化（L2O）技术与拟牛顿方法相结合，以学习参数化为神经网络的预条件矩阵；这些预条件矩阵比其他拟牛顿方法中的预条件矩阵更加灵活。与其他的L2O方法不同，LODO无需在训练任务分布上进行元训练，而是在测试任务上优化时即时学习优化，根据损失景观的局部特征自适应调整。理论上，我们证明了该优化器能在含噪损失景观中近似逆黑塞矩阵，并能表示多种逆黑塞矩阵。实验验证了该算法在噪声环境中的优化能力，同时表明表示逆黑塞矩阵的简单替代方案会降低性能。最后，我们使用该优化器训练了一个包含9.5万个参数的半真实深度神经网络，其速度与标准神经网络优化器相当。