There has been growing interest in high-order tensor methods for nonconvex optimization, with adaptive regularization, as they possess better/optimal worst-case evaluation complexity globally and faster convergence asymptotically. These algorithms crucially rely on repeatedly minimizing nonconvex multivariate Taylor-based polynomial sub-problems, at least locally. Finding efficient techniques for the solution of these sub-problems, beyond the second-order case, has been an open question. This paper proposes a second-order method, Quadratic Quartic Regularisation (QQR), for efficiently minimizing nonconvex quartically-regularized cubic polynomials, such as the AR$p$ sub-problem [3] with $p=3$. Inspired by [35], QQR approximates the third-order tensor term by a linear combination of quadratic and quartic terms, yielding (possibly nonconvex) local models that are solvable to global optimality. In order to achieve accuracy $\epsilon$ in the first-order criticality of the sub-problem, we show that the error in the QQR method decreases either linearly or by at least $\mathcal{O}(\epsilon^{4/3})$ for locally convex iterations, while in the sufficiently nonconvex case, by at least $\mathcal{O}(\epsilon)$; thus improving, on these types of iterations, the general cubic-regularization bound. Preliminary numerical experiments indicate that two QQR variants perform competitively with state-of-the-art approaches such as ARC (also known as AR$p$ with $p=2$), achieving either a lower objective value or iteration counts.

翻译：非凸优化中采用自适应正则化的高阶张量方法因其具有更好的/最优的最坏情况全局评估复杂度以及更快的渐近收敛性而受到越来越多的关注。这类算法的关键在于反复最小化基于泰勒展开的非凸多元多项式子问题（至少在局部范围内）。寻找除二阶情况外这些子问题的高效求解技术一直是一个悬而未决的问题。本文提出了一种名为二次-四次正则化（QQR）的二阶方法，用于高效最小化非凸四阶正则化三次多项式（例如AR$p$子问题[3]中$p=3$的情形）。受文献[35]启发，QQR通过将三阶张量项近似表示为二次项与四次项的线性组合，从而得到可全局最优求解的（可能非凸的）局部模型。为了在子问题的一阶临界性上达到精度$\epsilon$，我们证明了在局部凸迭代中QQR方法的误差呈线性下降或至少以$\mathcal{O}(\epsilon^{4/3})$的速度下降，而在充分非凸情形下则至少以$\mathcal{O}(\epsilon)$的速度下降；从而在这类迭代中改进了一般三次正则化方法的界。初步数值实验表明，两种QQR变体与ARC（即$p=2$时的AR$p$）等前沿方法相比具有竞争力，能够实现更低的函数值或更少的迭代次数。