Block coordinate descent (BCD), also known as nonlinear Gauss-Seidel, is a simple iterative algorithm for nonconvex optimization that sequentially minimizes the objective function in each block coordinate while the other coordinates are held fixed. We propose a version of BCD that, for block multi-convex and smooth objective functions under constraints, is guaranteed to converge to the stationary points with worst-case rate of convergence of $O((\log n)^{2}/n)$ for $n$ iterations, and a bound of $O(\epsilon^{-1}(\log \epsilon^{-1})^{2})$ for the number of iterations to achieve an $\epsilon$-approximate stationary point. Furthermore, we show that these results continue to hold even when the convex sub-problems are inexactly solved if the optimality gaps are uniformly summable against initialization. A key idea is to restrict the parameter search within a diminishing radius to promote stability of iterates. As an application, we provide an alternating least squares algorithm with diminishing radius for nonnegative CP tensor decomposition that converges to the stationary points of the reconstruction error with the same robust worst-case convergence rate and complexity bounds. We also experimentally validate our results with both synthetic and real-world data and demonstrate that using auxiliary search radius restriction can in fact improve the rate of convergence.

翻译：区块坐标下移( BCD), 也称为非线性 Gaus- Seidel, 是一个简单的非Convex优化的迭接算法, 其顺序将每个区块坐标的客观功能最小化, 而其他坐标则保持固定。 我们提议了一个 BCD 版本, 用于块状多convex 和 平滑客观功能, 在受限制的情况下, 保证它们会与固定点趋同, 最差的 $O (( log n)\ ⁇ 2} /n) 趋同率( ) 最差的 美元相趋同率( $O ( log n)\ ⁇ 2} 和 $ O( epsilon)-1} (log \ \ \ \ epslon =1} \\\\\\\\\\\\\\\\\\\\\\\\ 2} } 美元结合值( ) ) 美元) 。 我们用最差的 listalgalbilalityal) adalbilationalizalationalizalizal) 来显示Sqlationalizalizalizald, 和不甚高的Sqlationalvialdalviald 。 我们等同的正正正正正正正正变的 。 我们相向, 。 我们提供最差的 和最差的Salbilgalvialgalbalbildalgalbildalgalg 。 。 。 。 我们的Slation 。 。我们提供最差的 和最差的Sqralvialdaldaldalvialvialbalbal