In this paper, we propose new structured second-order methods and structured adaptive-gradient methods obtained by performing natural-gradient descent on structured parameter spaces. Natural-gradient descent is an attractive approach to design new algorithms in many settings such as gradient-free, adaptive-gradient, and second-order methods. Our structured methods not only enjoy a structural invariance but also admit a simple expression. Finally, we test the efficiency of our proposed methods on both deterministic non-convex problems and deep learning problems.

翻译：在本文中,我们提出了结构化的新型二级方法和结构化的适应性梯度方法,这些方法和结构化的适应性梯度方法是通过在结构化参数空间上进行自然梯度下降而获得的。自然梯度下降是设计许多环境中的新算法的有吸引力的方法,例如梯度、适应性梯度和二级方法。我们的结构化方法不仅具有结构性的偏差,而且承认一个简单的表达方式。最后,我们测试了我们所提议的方法在确定性非混凝土问题和深层学习问题上的效率。