Nonsmooth composite optimization with orthogonality constraints has a broad spectrum of applications in statistical learning and data science. However, this problem is generally challenging to solve due to its non-convex and non-smooth nature. Existing solutions are limited by one or more of the following restrictions: (i) they are full gradient methods that require high computational costs in each iteration; (ii) they are not capable of solving general nonsmooth composite problems; (iii) they are infeasible methods and can only achieve the feasibility of the solution at the limit point; (iv) they lack rigorous convergence guarantees; (v) they only obtain weak optimality of critical points. In this paper, we propose \textit{\textbf{OBCD}}, a new Block Coordinate Descent method for solving general nonsmooth composite problems under Orthogonality constraints. \textit{\textbf{OBCD}} is a feasible method with low computation complexity footprints. In each iteration, our algorithm updates $k$ rows of the solution matrix ($k\geq2$ is a parameter) to preserve the constraints. Then, it solves a small-sized nonsmooth composite optimization problem under orthogonality constraints either exactly or approximately. We demonstrate that any exact block-$k$ stationary point is always an approximate block-$k$ stationary point, which is equivalent to the critical stationary point. We are particularly interested in the case where $k=2$ as the resulting subproblem reduces to a one-dimensional nonconvex problem. We propose a breakpoint searching method and a fifth-order iterative method to solve this problem efficiently and effectively. We also propose two novel greedy strategies to find a good working set to further accelerate the convergence of \textit{\textbf{OBCD}}. Finally, we have conducted extensive experiments on several tasks to demonstrate the superiority of our approach.

翻译：正交约束下的非光滑复合优化在统计学习和数据科学中具有广泛应用。然而，由于问题的非凸性和非光滑性，求解此类问题通常具有挑战性。现有方法存在以下一个或多个局限性：(i) 采用全梯度方法导致每次迭代计算成本高；(ii) 无法求解一般非光滑复合问题；(iii) 仅作为不可行方法在极限点处满足约束可行性；(iv) 缺乏严格的收敛性保证；(v) 仅能获得弱最优性的临界点。本文提出\textbf{\textit{OBCD}}——一种求解正交约束下一般非光滑复合问题的新型块坐标下降方法。该方法作为一种可行算法，具有低计算复杂度特性。每次迭代中，算法更新解矩阵的$k$行（$k\geq2$为参数）以保持约束条件，随后精确或近似求解一个正交约束下的小规模非光滑复合优化子问题。我们证明任意精确块-$k$驻点必然是近似块-$k$驻点，等价于临界驻点。特别关注$k=2$的情形，此时子问题退化为单变量非凸问题。为此提出断点搜索法与五阶迭代法高效求解该问题。此外，引入两种新型贪婪策略寻找优质工作集以加速\textit{\textbf{OBCD}}收敛。最后，通过多项任务的广泛实验验证了本方法的优越性。