Privacy and Byzantine resilience (BR) are two crucial requirements of modern-day distributed machine learning. The two concepts have been extensively studied individually but the question of how to combine them effectively remains unanswered. This paper contributes to addressing this question by studying the extent to which the distributed SGD algorithm, in the standard parameter-server architecture, can learn an accurate model despite (a) a fraction of the workers being malicious (Byzantine), and (b) the other fraction, whilst being honest, providing noisy information to the server to ensure differential privacy (DP). We first observe that the integration of standard practices in DP and BR is not straightforward. In fact, we show that many existing results on the convergence of distributed SGD under Byzantine faults, especially those relying on $(\alpha,f)$-Byzantine resilience, are rendered invalid when honest workers enforce DP. To circumvent this shortcoming, we revisit the theory of $(\alpha,f)$-BR to obtain an approximate convergence guarantee. Our analysis provides key insights on how to improve this guarantee through hyperparameter optimization. Essentially, our theoretical and empirical results show that (1) an imprudent combination of standard approaches to DP and BR might be fruitless, but (2) by carefully re-tuning the learning algorithm, we can obtain reasonable learning accuracy while simultaneously guaranteeing DP and BR.

翻译：隐私和拜占庭复原力(BR)是现代分布式机器学习的两个关键要求。这两个概念已经单独进行了广泛研究,但如何有效结合这两个概念的问题仍然没有得到回答。本文通过研究标准参数-服务器架构中分布的SGD算法(标准参数-服务器架构中分布的SGD算法)在多大程度上在诚实的工人执行DP时会变得无效而有助于解决这一问题,尽管:(a) 部分工人恶意(Byzantine),和(b) 另一部分工人是诚实的,向服务器提供噪音信息以确保差异性隐私(DP ) 。我们首先发现,DP和BR的标准做法的整合并非直截了当。事实上,我们表明,在Byzantine断层断层断层断层下,特别是依赖$(alpha,f)$-Byzantine的SGD算法(SGD)的分布式算法(SGDGD)的现有许多结果,特别是那些依靠美元(Byzantine Revitine)的算法($)的算法(BR)当诚实的工人执行DP时,而同时显示,我们通过学习标准的BRBR的精准方法可以正确性地学习到DDDDDD(BR)和BRBR),我们通过学习获得标准的组合。