In this paper, we develop two new randomized block-coordinate optimistic gradient algorithms to approximate a solution of nonlinear equations, which are called root-finding problems. Our first algorithm is non-accelerated with constant stepsizes, and achieves $\mathcal{O}(1/k)$ best-iterate convergence rate on $\mathbb{E}[ \Vert Gx^k\Vert^2]$ when the underlying operator $G$ is Lipschitz continuous and the equation $Gx = 0$ admits a weak Minty solution, where $\mathbb{E}[\cdot]$ is the expectation and $k$ is the iteration counter. Our second method is a new accelerated randomized block-coordinate optimistic gradient algorithm. We establish both $\mathcal{O}(1/k^2)$ and $o(1/k^2)$ last-iterate convergence rates on both $\mathbb{E}[ \Vert Gx^k\Vert^2]$ and $\mathbb{E}[ \Vert x^{k+1} - x^{k}\Vert^2]$ for this algorithm under the co-coerciveness of $G$. Then, we apply our methods to a class of finite-sum nonlinear inclusions which covers various applications in machine learning and statistical learning, especially in federated learning and network optimization. We obtain two new federated learning-type algorithms for this problem class with rigorous convergence rate guarantees.
翻译:本文提出了两种新的随机块坐标优化梯度算法,用于逼近非线性方程的解,这类问题称为求根问题。第一种算法采用恒定步长且无加速,在底层算子$G$满足Lipschitz连续性且方程$Gx=0$存在弱Minty解时,该算法在$\mathbb{E}[ \Vert Gx^k\Vert^2]$上达到$\mathcal{O}(1/k)$的最优迭代收敛率,其中$\mathbb{E}[\cdot]$表示期望,$k$为迭代次数。第二种方法是一种新型加速随机块坐标优化梯度算法。在$G$满足共强制性的条件下,我们证明了该算法在$\mathbb{E}[ \Vert Gx^k\Vert^2]$和$\mathbb{E}[ \Vert x^{k+1} - x^{k}\Vert^2]$上分别具有$\mathcal{O}(1/k^2)$和$o(1/k^2)$的最终迭代收敛率。随后,我们将所提方法应用于一类涵盖机器学习和统计学习(尤其是联邦学习与网络优化)中多种应用的有限和型非线性包含问题,并针对此类问题给出了两种具有严格收敛率保证的新型联邦学习算法。