We consider minimizing a smooth function subject to a summation constraint over its variables. By exploiting a connection between the greedy 2-coordinate update for this problem and equality-constrained steepest descent in the 1-norm, we give a convergence rate for greedy selection under a proximal Polyak-Lojasiewicz assumption that is faster than random selection and independent of the problem dimension $n$. We then consider minimizing with both a summation constraint and bound constraints, as arises in the support vector machine dual problem. Existing greedy rules for this setting either guarantee trivial progress only or require $O(n^2)$ time to compute. We show that bound- and summation-constrained steepest descent in the L1-norm guarantees more progress per iteration than previous rules and can be computed in only $O(n \log n)$ time.
翻译:我们考虑在变量求和约束下最小化光滑函数的问题。通过利用该问题中贪婪2坐标更新与1-范数等式约束最速下降之间的联系,我们在近端Polyak-Lojasiewicz假设下给出了贪婪选择的收敛速率,该速率优于随机选择且与问题维度$n$无关。随后我们进一步考虑同时具有求和约束和边界约束的最小化问题,此类问题出现在支持向量机对偶问题中。现有针对该场景的贪婪规则要么仅能保证平凡的进展,要么需要$O(n^2)$时间进行计算。我们证明L1-范数下带边界与求和约束的最速下降法每次迭代能获得比现有规则更优的进展,且仅需$O(n \log n)$时间即可完成计算。