This paper develops a fully distributed differentially-private learning algorithm to solve nonsmooth optimization problems. We distribute the Alternating Direction Method of Multipliers (ADMM) to comply with the distributed setting and employ an approximation of the augmented Lagrangian to handle nonsmooth objective functions. Furthermore, we ensure zero-concentrated differential privacy (zCDP) by perturbing the outcome of the computation at each agent with a variance-decreasing Gaussian noise. This privacy-preserving method allows for better accuracy than the conventional $(\epsilon, \delta)$-DP and stronger guarantees than the more recent R\'enyi-DP. The developed fully distributed algorithm has a competitive privacy accuracy trade-off and handles nonsmooth and non-necessarily strongly convex problems. We provide complete theoretical proof for the privacy guarantees and the convergence of the algorithm to the exact solution. We also prove under additional assumptions that the algorithm converges in linear time. Finally, we observe in simulations that the developed algorithm outperforms all of the existing methods.

翻译：本文提出了一种完全分布式的差分隐私学习算法，用于解决非光滑优化问题。我们通过分布交替方向乘子法（ADMM）以适应分布式场景，并利用增广拉格朗日函数的近似处理非光滑目标函数。此外，我们通过向每个智能体的计算结果添加方差递减的高斯噪声来实现零集中差分隐私（zCDP）。这种隐私保护方法相比传统的$(\epsilon, \delta)$-DP具有更高的精度，且比更近期的Rényi-DP提供更强的保障。所开发的完全分布式算法在隐私-精度权衡上具有竞争力，并能处理非光滑且不必强凸的问题。我们提供了隐私保证以及算法收敛到精确解的完整理论证明，并在额外假设下证明了算法具有线性收敛速度。最后，仿真结果表明，该算法优于所有现有方法。