High-order tensor methods that employ Taylor-based local models (of degree $p\ge 3$) within adaptive regularization frameworks have been recently proposed for both convex and nonconvex optimization problems. They have been shown to have superior, and even optimal, worst-case global convergence rates and local rates compared to Newton's method. Finding rigorous and efficient techniques for minimizing the Taylor polynomial sub-problems remains a challenging aspect for these algorithms. Ahmadi et al. recently introduced a tensor method based on sum-of-squares (SoS) reformulations, so that each Taylor polynomial sub-problem in their approach can be tractably minimized using semidefinite programming (SDP); however, the global convergence and complexity of their method have not been addressed for general nonconvex problems. This paper introduces an algorithmic framework that combines the Sum of Squares (SoS) Taylor model with adaptive regularization techniques for nonconvex smooth optimization problems. Each iteration minimizes an SoS Taylor model, offering a polynomial cost per iteration. For general nonconvex functions, the worst-case evaluation complexity bound is $\mathcal{O}(\epsilon^{-2})$, while for strongly convex functions, an improved evaluation complexity bound of $\mathcal{O}(\epsilon^{-\frac{1}{p}})$ is established. To the best of our knowledge, this is the first global rate analysis for an adaptive regularization algorithm with a tractable high-order sub-problem in nonconvex smooth optimization, opening the way for further improvements.

翻译：近年来，在自适应正则化框架下采用基于泰勒局部模型（阶数$p\ge 3$）的高阶张量方法已被提出，用于解决凸优化和非凸优化问题。与牛顿法相比，这些方法已被证明具有更优甚至最优的最坏情形全局收敛速度和局部收敛速度。然而，如何找到严格且高效的方法来最小化泰勒多项式子问题，仍是这些算法面临的挑战。Ahmadi等人近期提出了一种基于平方和（SoS）重构的张量方法，使得其每个泰勒多项式子问题可通过半定规划（SDP）有效求解；但该方法在一般非凸问题上的全局收敛性与复杂度尚未得到解决。本文提出一种算法框架，将平方和（SoS）泰勒模型与自适应正则化技术相结合，用于非凸光滑优化问题。每次迭代最小化一个SoS泰勒模型，每次迭代的计算复杂度呈多项式量级。对于一般非凸函数，最坏情形评估复杂度界为$\mathcal{O}(\epsilon^{-2})$，而对于强凸函数，则建立了改进的评估复杂度界$\mathcal{O}(\epsilon^{-\frac{1}{p}})$。据我们所知，这是首个在非凸光滑优化中针对具有可解高阶子问题的自适应正则化算法进行全局速率分析的工作，为后续改进开辟了道路。