Alternating Direction Method of Multipliers (ADMM) is a popular method for solving large-scale Machine Learning problems. Stochastic ADMM was proposed to reduce the per iteration computational complexity, which is more suitable for big data problems. Recently, variance reduction techniques have been integrated with stochastic ADMM in order to get a faster convergence rate, such as SAG-ADMM and SVRG-ADMM. However, their convergence rate is still suboptimal w.r.t the smoothness constant. In this paper, we propose an accelerated stochastic ADMM algorithm with variance reduction, which enjoys a faster convergence than all the existing stochastic ADMM algorithms. We theoretically analyse its convergence rate and show its dependence on the smoothness constant is optimal. We also empirically validate its effectiveness and show its priority over other stochastic ADMM algorithms.

翻译：交替方向乘子法（ADMM）是求解大规模机器学习问题的常用方法。为降低每次迭代的计算复杂度，学者提出了随机ADMM，使其更适用于大数据问题。近期，为获得更快的收敛速度，方差缩减技术被引入随机ADMM，例如SAG-ADMM和SVRG-ADMM。然而，这些方法在光滑常数方面的收敛率仍非最优。本文提出一种加速方差缩减的随机ADMM算法，其收敛速度优于现有所有随机ADMM算法。我们从理论上分析了该算法的收敛率，证明其关于光滑常数的依赖性达到最优，并通过实验验证了该算法的有效性及其相较于其他随机ADMM算法的优越性。