We consider the fundamental task of optimising a real-valued function defined in a potentially high-dimensional Euclidean space, such as the loss function in many machine-learning tasks or the logarithm of the probability distribution in statistical inference. We use Riemannian geometry notions to redefine the optimisation problem of a function on the Euclidean space to a Riemannian manifold with a warped metric, and then find the function's optimum along this manifold. The warped metric chosen for the search domain induces a computational friendly metric-tensor for which optimal search directions associated with geodesic curves on the manifold becomes easier to compute. Performing optimization along geodesics is known to be generally infeasible, yet we show that in this specific manifold we can analytically derive Taylor approximations up to third-order. In general these approximations to the geodesic curve will not lie on the manifold, however we construct suitable retraction maps to pull them back onto the manifold. Therefore, we can efficiently optimize along the approximate geodesic curves. We cover the related theory, describe a practical optimization algorithm and empirically evaluate it on a collection of challenging optimisation benchmarks. Our proposed algorithm, using 3rd-order approximation of geodesics, tends to outperform standard Euclidean gradient-based counterparts in term of number of iterations until convergence.

翻译：我们考虑优化一个定义在潜在高维欧几里得空间中的实值函数的基本任务，例如许多机器学习任务中的损失函数或统计推断中概率分布的对数。我们利用黎曼几何概念，将欧几里得空间上的函数优化问题重新定义为具有扭曲度量的黎曼流形上的优化，并沿着该流形寻找函数的最优解。为搜索域选择的扭曲度量引入了一种计算友好的度量张量，使得与流形上测地线相关的最优搜索方向更易于计算。已知沿测地线进行优化通常不可行，但我们证明在这个特定流形上，我们可以解析地推导出高达三阶的泰勒近似。通常，这些对测地线的近似不会落在流形上，然而我们构造了合适的回缩映射将它们拉回流形。因此，我们可以沿近似测地线高效优化。我们阐述了相关理论，描述了一个实用的优化算法，并在一个具有挑战性的优化基准测试集上进行了实证评估。我们提出的算法使用三阶测地线近似，在收敛所需的迭代次数方面往往优于标准的基于欧几里得梯度的对应方法。