The Newton, Gauss--Newton and Levenberg--Marquardt methods all use the first derivative of a vector function (the Jacobian) to minimise its sum of squares. When the Jacobian matrix is ill-conditioned, the function varies much faster in some directions than others and the space of possible improvement in sum of squares becomes a long narrow ellipsoid in the linear model. This means that even a small amount of nonlinearity in the problem parameters can cause a proposed point far down the long axis of the ellipsoid to fall outside of the actual curved valley of improved values, even though it is quite nearby. This paper presents a differential equation that `follows' these valleys, based on the technique of geodesic acceleration, which itself provides a 2$^\mathrm{nd}$ order improvement to the Levenberg--Marquardt iteration step. Higher derivatives of this equation are computed that allow $n^\mathrm{th}$ order improvements to the optimisation methods to be derived. These higher-order accelerated methods up to 4$^\mathrm{th}$ order are tested numerically and shown to provide substantial reduction of both number of steps and computation time.

翻译：牛顿法、高斯-牛顿法和莱文贝格-马夸特法均利用向量函数的一阶导数（雅可比矩阵）来最小化其平方和。当雅可比矩阵病态时，函数在某些方向上的变化远快于其他方向，平方和可能改进的空间在线性模型中变为一个狭长的椭球体。这意味着问题参数即使仅存在少量非线性，也会导致位于椭球体长轴远端附近的提议点，即便距离实际改进值的弯曲谷地很近，仍可能落于其外。本文基于测地线加速技术，提出了一种“沿谷地行进”的微分方程，该技术本身即为莱文贝格-马夸特迭代步提供了二阶改进。通过计算该方程的高阶导数，可推导出优化方法的n阶改进。对这些高达四阶的高阶加速方法进行了数值测试，结果表明其能够显著减少迭代步数与计算时间。