We propose an accelerated block proximal linear framework with adaptive momentum (ABPL$^+$) for nonconvex and nonsmooth optimization. We analyze the potential causes of the extrapolation step failing in some algorithms, and resolve this issue by enhancing the comparison process that evaluates the trade-off between the proximal gradient step and the linear extrapolation step in our algorithm. Furthermore, we extends our algorithm to any scenario involving updating block variables with positive integers, allowing each cycle to randomly shuffle the update order of the variable blocks. Additionally, under mild assumptions, we prove that ABPL$^+$ can monotonically decrease the function value without strictly restricting the extrapolation parameters and step size, demonstrates the viability and effectiveness of updating these blocks in a random order, and we also more obviously and intuitively demonstrate that the derivative set of the sequence generated by our algorithm is a critical point set. Moreover, we demonstrate the global convergence as well as the linear and sublinear convergence rates of our algorithm by utilizing the Kurdyka-Lojasiewicz (K{\L}) condition. To enhance the effectiveness and flexibility of our algorithm, we also expand the study to the imprecise version of our algorithm and construct an adaptive extrapolation parameter strategy, which improving its overall performance. We apply our algorithm to multiple non-negative matrix factorization with the $\ell_0$ norm, nonnegative tensor decomposition with the $\ell_0$ norm, and perform extensive numerical experiments to validate its effectiveness and efficiency.
翻译:我们提出了一种具有自适应动量的加速块近端线性框架(ABPL$^+$)用于非凸非光滑优化。我们分析了某些算法中外推步骤失败的潜在原因,并通过增强算法中评估近端梯度步骤与线性外推步骤之间权衡的比较过程解决了这一问题。此外,我们将算法推广到任意涉及更新正整数值块变量的场景,允许每个周期随机打乱变量块的更新顺序。同时,在温和假设下,我们证明ABPL$^+$可以在不严格限制外推参数和步长的情况下单调减小函数值,展示了随机顺序更新这些块的可行性和有效性,并且我们更明显直观地证明了算法生成序列的导集是临界点集。此外,我们利用Kurdyka-Lojasiewicz(KŁ)条件证明了算法的全局收敛性以及线性和次线性收敛速率。为了提高算法的有效性和灵活性,我们还研究了算法的不精确版本,并构建了自适应外推参数策略,从而提升了整体性能。我们将算法应用于包含$\ell_0$范数的多重非负矩阵分解和包含$\ell_0$范数的非负张量分解,并进行了大量数值实验,以验证其有效性和效率。