Gradient dominance property is a condition weaker than strong convexity, yet it sufficiently ensures global convergence for first-order methods even in non-convex optimization. This property finds application in various machine learning domains, including matrix decomposition, linear neural networks, and policy-based reinforcement learning (RL). In this paper, we study the stochastic homogeneous second-order descent method (SHSODM) for gradient-dominated optimization with $\alpha \in [1, 2]$ based on a recently proposed homogenization approach. Theoretically, we show that SHSODM achieves a sample complexity of $O(\epsilon^{-7/(2 \alpha) +1})$ for $\alpha \in [1, 3/2)$ and $\tilde{O}(\epsilon^{-2/\alpha})$ for $\alpha \in [3/2, 2]$. We further provide a SHSODM with a variance reduction technique enjoying an improved sample complexity of $O( \epsilon ^{-( 7-3\alpha ) /( 2\alpha )})$ for $\alpha \in [1,3/2)$. Our results match the state-of-the-art sample complexity bounds for stochastic gradient-dominated optimization without \emph{cubic regularization}. Since the homogenization approach only relies on solving extremal eigenvector problems instead of Newton-type systems, our methods gain the advantage of cheaper iterations and robustness in ill-conditioned problems. Numerical experiments on several RL tasks demonstrate the efficiency of SHSODM compared to other off-the-shelf methods.

翻译：梯度主导性质是比强凸性更弱的条件，但即使对于非凸优化，它也足以确保一阶方法的全局收敛性。该性质应用于多种机器学习领域，包括矩阵分解、线性神经网络以及基于策略的强化学习（RL）。本文基于近期提出的同质化方法，研究随机同质化二阶下降法（SHSODM）在$\alpha \in [1, 2]$的梯度主导优化中的应用。理论上，我们证明SHSODM在$\alpha \in [1, 3/2)$时达到$O(\epsilon^{-7/(2\alpha)+1})$的样本复杂度，在$\alpha \in [3/2, 2]$时达到$\tilde{O}(\epsilon^{-2/\alpha})$的样本复杂度。我们进一步提出结合方差缩减技术的SHSODM，在$\alpha \in [1,3/2)$时实现改进的$O(\epsilon^{-(7-3\alpha)/(2\alpha)})$样本复杂度。我们的结果与无需\emph{三次正则化}的随机梯度主导优化最优样本复杂度界相匹配。由于同质化方法仅依赖于求解极端特征向量问题而非牛顿型系统，我们的方法在病态问题中具有迭代代价更小和鲁棒性更强的优势。在多个强化学习任务上的数值实验表明，SHSODM相较于其他现成方法具有高效性。