We consider the fundamental task of optimizing a real-valued function defined in a potentially high-dimensional Euclidean space, such as the loss function in many machine-learning tasks or the logarithm of the probability distribution in statistical inference. We use the warped Riemannian geometry notions to redefine the optimisation problem of a function on Euclidean space to a Riemannian manifold with a warped metric, and then find the function's optimum along this manifold. The warped metric chosen for the search domain induces a computational friendly metric-tensor for which optimal search directions associate with geodesic curves on the manifold becomes easier to compute. Performing optimization along geodesics is known to be generally infeasible, yet we show that in this specific manifold we can analytically derive Taylor approximations up to third-order. In general these approximations to the geodesic curve will not lie on the manifold, however we construct suitable retraction maps to pull them back onto the manifold. Therefore, we can efficiently optimize along the approximate geodesic curves. We cover the related theory, describe a practical optimization algorithm and empirically evaluate it on a collection of challenging optimisation benchmarks. Our proposed algorithm, using third-order approximation of geodesics, outperforms standard Euclidean gradient-based counterparts in term of number of iterations until convergence and an alternative method for Hessian-based optimisation routines.

翻译：我们考虑优化定义在高维欧几里得空间中的实值函数这一基本任务，例如许多机器学习任务中的损失函数或统计推断中的概率分布对数函数。利用扭曲黎曼几何的概念，我们将欧几里得空间上的函数优化问题重新定义为具有扭曲度量的黎曼流形上的问题，并沿此流形寻找函数的最优值。为搜索域选择的扭曲度量生成一种计算友好的度量张量，使得与流形上测地线相关的优化搜索方向更易于计算。沿测地线执行优化通常被认为是不可行的，但我们证明在此特定流形上可以解析推导出高达三阶的泰勒近似。虽然这些测地线曲线近似通常不在流形上，但我们构造了合适的回缩映射将其拉回至流形。因此，我们可以沿近似测地线高效优化。本文覆盖相关理论，描述实用优化算法，并在具有挑战性的优化基准集上对其进行实证评估。我们提出的算法利用测地线三阶近似，在达到收敛所需的迭代次数方面优于标准欧几里得梯度对应方法，以及一种基于海森矩阵的优化替代方法。