The recent deployment of multi-agent networks has enabled the distributed solution of learning problems, where agents cooperate to train a global model without sharing their local, private data. This work specifically targets some prevalent challenges inherent to distributed learning: (i) online training, i.e., the local data change over time; (ii) asynchronous agent computations; (iii) unreliable and limited communications; and (iv) inexact local computations. To tackle these challenges, we apply the Distributed Operator Theoretical (DOT) version of the Alternating Direction Method of Multipliers (ADMM), which we call "DOT-ADMM". We prove that if the DOT-ADMM operator is metric subregular, then it converges with a linear rate for a large class of (not necessarily strongly) convex learning problems toward a bounded neighborhood of the optimal time-varying solution, and characterize how such neighborhood depends on (i)-(iv). We first derive an easy-to-verify condition for ensuring the metric subregularity of an operator, followed by tutorial examples on linear and logistic regression problems. We corroborate the theoretical analysis with numerical simulations comparing DOT-ADMM with other state-of-the-art algorithms, showing that only the proposed algorithm exhibits robustness to (i)-(iv).

翻译：多代理网络的近期部署使得学习问题的分布式求解得以实现，其中各代理通过协作训练全局模型而无需共享本地私有数据。本工作专门针对分布式学习中普遍存在的若干挑战：(i) 在线训练（即本地数据随时间变化）；(ii) 异步代理计算；(iii) 不可靠且受限的通信；以及(iv) 非精确本地计算。为应对这些挑战，我们应用了交替方向乘子法（ADMM）的分布式算子理论（DOT）版本，称之为“DOT-ADMM”。我们证明：若DOT-ADMM算子满足度量次正则性，则对于一大类（不必强）凸学习问题，该算子将以线性收敛速率收敛至最优时变解的有界邻域内，并刻画该邻域如何依赖于(i)-(iv)。我们首先推导确保算子满足度量次正则性的易验证条件，随后给出关于线性回归与逻辑回归问题的示例教程。通过数值仿真将DOT-ADMM与其他前沿算法进行对比，理论分析得到验证：仅所提算法对(i)-(iv)展现出鲁棒性。