The classical non-greedy algorithm (NGA) and the recently proposed proximal alternating minimization method with extrapolation (PAMe) for $L_1$-norm PCA are revisited and their finite-step convergence are studied. It is first shown that NGA can be interpreted as a conditional subgradient or an alternating maximization method. By recognizing it as a conditional subgradient, we prove that the iterative points generated by the algorithm will be constant in finitely many steps under a certain full-rank assumption; such an assumption can be removed when the projection dimension is one. By treating the algorithm as an alternating maximization, we then prove that the objective value will be fixed after at most $\left\lceil\frac{F^{\max}}{\tau_0} \right\rceil$ steps, where the stopping point satisfies certain optimality conditions. Then, a slight modification of NGA with improved convergence properties is analyzed. It is shown that the iterative points generated by the modified algorithm will not change after at most $\left\lceil\frac{2F^{\max}}{\tau} \right\rceil$ steps; furthermore, the stopping point satisfies certain optimality conditions if the proximal parameter $\tau$ is small enough. For PAMe, it is proved that the sign variable will remain constant after finitely many steps and the algorithm can output a point satisfying certain optimality condition, if the parameters are small enough and a full rank assumption is satisfied. Moreover, if there is no proximal term on the projection matrix related subproblem, then the iterative points generated by this modified algorithm will not change after at most $\left\lceil \frac{4F^{\max}}{\tau(1-\gamma)} \right\rceil$ steps and the stopping point also satisfies certain optimality conditions, provided similar assumptions as those for PAMe. The full rank assumption can be removed when the projection dimension is one.
翻译:针对$L_1$范数主成分分析(PCA)问题,本文重新审视了经典的非贪婪算法(NGA)及近期提出的带外推近端交替极小化方法(PAMe),并研究了其有限步收敛行为。首先证明NGA可被解释为条件次梯度法或交替极大化方法。基于条件次梯度视角,我们证明在满足特定满秩假设时,算法生成的迭代点将在有限步内保持恒定;当投影维数为一维时,该假设可被移除。通过将其视为交替极大化方法,进一步证明目标函数值在至多$\left\lceil\frac{F^{\max}}{\tau_0} \right\rceil$步后将固定不变,且终止点满足特定最优性条件。随后分析了一种经轻微改进的NGA变体,其收敛性质更优。实验证明,改进算法的迭代点在至多$\left\lceil\frac{2F^{\max}}{\tau} \right\rceil$步后不再变化;当近端参数$\tau$足够小时,终止点满足特定最优性条件。对于PAMe,我们证明在参数足够小且满足满秩假设时,符号变量在有限步后保持恒定,算法可输出满足特定最优性条件的解点。此外,若投影矩阵相关子问题中不含近端项,则在类似PAMe的假设条件下,改进算法的迭代点在至多$\left\lceil \frac{4F^{\max}}{\tau(1-\gamma)} \right\rceil$步后停止变化,且终止点同样满足特定最优性条件。当投影维数为一维时,该满秩假设可被移除。