Natural-gradient descent (NGD) on structured parameter spaces (e.g., low-rank covariances) is computationally challenging due to difficult Fisher-matrix computations. We address this issue by using \emph{local-parameter coordinates} to obtain a flexible and efficient NGD method that works well for a wide-variety of structured parameterizations. We show four applications where our method (1) generalizes the exponential natural evolutionary strategy, (2) recovers existing Newton-like algorithms, (3) yields new structured second-order algorithms via matrix groups, and (4) gives new algorithms to learn covariances of Gaussian and Wishart-based distributions. We show results on a range of problems from deep learning, variational inference, and evolution strategies. Our work opens a new direction for scalable structured geometric methods.

翻译：在结构化参数空间(例如低级共变)上的自然渐变(NGD)在计算上具有挑战性,因为很难进行渔业矩阵的计算。我们通过使用 emph{当地参数坐标 来解决这个问题。我们通过使用 emph{当地参数坐标 来获得灵活有效的NGD 方法,该方法对于结构化参数化的广度非常有效。我们展示了四个应用方法:(1) 概括指数性自然进化战略,(2) 恢复现有的牛顿式算法,(3) 通过矩阵组产生新的结构化第二序列算法,(4) 提供新的算法,学习高斯和Wishart分布法的共变。我们展示了从深层次学习、变异推论和进化战略等一系列问题的结果。我们的工作为可缩放结构化的几何方法开辟了新的方向。