Natural-gradient descent on structured parameter spaces (e.g., low-rank covariances) is computationally challenging due to complicated inverse Fisher-matrix computations. We address this issue for optimization, inference, and search problems by using \emph{local-parameter coordinates}. Our method generalizes an existing evolutionary-strategy method, recovers Newton and Riemannian-gradient methods as special cases, and also yields new tractable natural-gradient algorithms for learning flexible covariance structures of Gaussian and Wishart-based distributions. We show results on a range of applications on deep learning, variational inference, and evolution strategies. Our work opens a new direction for scalable structured geometric methods via local parameterizations.

翻译：在结构化参数空间(例如低位共变)上的自然梯度下降由于复杂的反向渔业矩阵计算而具有计算上的挑战性。我们通过使用 \ emph{ 本地参数坐标来解决这个问题,以便优化、推断和搜索问题。我们的方法概括了现有的进化战略方法,将牛顿和里曼梯度方法作为特例加以恢复,并产生了新的可移动的自然梯度算法,用于学习高萨和Wishart分布的灵活共变结构。我们展示了在深层学习、变异推断和进化战略方面的一系列应用结果。我们的工作为通过本地参数化的可缩放结构化几何方法开辟了新的方向。