Stochastic versions of the alternating direction method of multiplier (ADMM) and its variants play a key role in many modern large-scale machine learning problems. In this work, we introduce a unified algorithmic framework called generalized stochastic ADMM and investigate their continuous-time analysis. The generalized framework widely includes many stochastic ADMM variants such as standard, linearized and gradient-based ADMM. Our continuous-time analysis provides us with new insights into stochastic ADMM and variants, and we rigorously prove that under some proper scaling, the trajectory of stochastic ADMM weakly converges to the solution of a stochastic differential equation with small noise. Our analysis also provides a theoretical explanation of why the relaxation parameter should be chosen between 0 and 2.

翻译：交替方向乘子法（ADMM）及其变体的随机化版本在现代大规模机器学习问题中起着关键作用。本文提出一个统一的算法框架——广义随机ADMM，并对其进行连续时间分析。该广义框架广泛涵盖多种随机ADMM变体，包括标准形式、线性化形式和基于梯度的ADMM。通过连续时间分析，我们获得了对随机ADMM及其变体的新认知，并严格证明：在适当尺度化条件下，随机ADMM的轨迹弱收敛于具有小噪声的随机微分方程的解。该分析同时也为松弛参数为何应取值于0到2之间提供了理论解释。