In this work, we present a generic step-size choice for the ADMM type proximal algorithms. It admits a closed-form expression and is theoretically optimal with respect to a worst-case convergence rate bound. It is simply given by the ratio of Euclidean norms of the dual and primal solutions, i.e., $ ||{\lambda}^\star|| / ||{x}^\star||$. Numerical tests show that its practical performance is near-optimal in general. The only challenge is that such a ratio is not known a priori and we provide two strategies to address it. The derivation of our step-size choice is based on studying the fixed-point structure of ADMM using the proximal operator. However, we demonstrate that the classical proximal operator definition contains an input scaling issue. This leads to a scaled step-size optimization problem which would yield a false solution. Such an issue is naturally avoided by our proposed new definition of the proximal operator. A series of its properties is established.
翻译:本文提出了一种适用于ADMM型近端算法的通用步长选择方法。该步长具有闭式表达式,且在理论上基于最坏情况收敛速率界达到最优,其形式简单地由对偶解与原始解的欧几里得范数之比给出,即$ ||{\lambda}^\star|| / ||{x}^\star||$。数值实验表明,该步长在实际应用中通常具有近似最优性能。唯一挑战在于此类比值无法先验获知,为此我们提出了两种应对策略。该步长选择的推导基于利用近端算子研究ADMM的固定点结构。然而,我们证明经典近端算子定义存在输入缩放问题,这会导致一个产生虚假解的有缩放步长优化问题。我们提出的近端算子新定义自然规避了该问题,并建立了其一系列性质。