Optimizing strongly convex functions subject to linear constraints is a fundamental problem with numerous applications. In this work, we propose a block (accelerated) randomized Bregman-Kaczmarz method that only uses a block of constraints in each iteration to tackle this problem. We consider a dual formulation of this problem in order to deal in an efficient way with the linear constraints. Using convex tools, we show that the corresponding dual function satisfies the Polyak-Lojasiewicz (PL) property, provided that the primal objective function is strongly convex and verifies additionally some other mild assumptions. However, adapting the existing theory on coordinate descent methods to our dual formulation can only give us sublinear convergence results in the dual space. In order to obtain convergence results in some criterion corresponding to the primal (original) problem, we transfer our algorithm to the primal space, which combined with the PL property allows us to get linear convergence rates. More specifically, we provide a theoretical analysis of the convergence of our proposed method under different assumptions on the objective and demonstrate in the numerical experiments its superior efficiency and speed up compared to existing methods for the same problem.
翻译:优化受线性约束的强凸函数是一个具有众多应用的基础性问题。本文提出了一种块(加速)随机Bregman-Kaczmarz方法,该方法在每次迭代中仅使用一个约束块来处理该问题。我们考虑该问题的对偶形式,以高效处理线性约束。利用凸分析工具,我们证明相应的对偶函数满足Polyak-Lojasiewicz(PL)性质,前提是原始目标函数为强凸函数,并满足其他一些温和假设。然而,将现有的坐标下降法理论直接应用于我们的对偶形式,仅能给出对偶空间中的次线性收敛结果。为了获得对应于原始问题的某些准则下的收敛结果,我们将算法转换到原始空间,结合PL性质,从而得到线性收敛速率。具体而言,我们在目标函数的不同假设下对所提方法的收敛性进行了理论分析,并通过数值实验证明,该方法在解决同一问题时相较于现有方法具有更优的效率和加速效果。