In many distributed learning setups such as federated learning (FL), client nodes at the edge use individually collected data to compute local gradients and send them to a central master server. The master server then aggregates the received gradients and broadcasts the aggregation to all clients, with which the clients can update the global model. In this paper, we consider multi-server federated learning with secure aggregation and unreliable communication links. We first define a threat model using Shannon's information-theoretic security framework and propose a novel scheme called Lagrange Coding with Mask (LCM), which divides the servers into groups and uses Coding and Masking techniques. LCM can achieve a trade-off between the uplink and downlink communication loads by adjusting the number of servers in each group. Furthermore, we derive the lower bounds of the uplink and downlink communication loads, respectively, and prove that LCM achieves the optimal uplink communication load, which is unrelated to the number of collusion clients.
翻译:摘要:在许多分布式学习场景中(如联邦学习),边缘客户端利用各自收集的数据计算本地梯度,并将其发送至中心主服务器。主服务器聚合接收到的梯度后向所有客户端广播聚合结果,客户端据此更新全局模型。本文考虑基于不可靠通信链路的多服务器联邦学习与安全聚合问题。我们首先利用香农信息论安全框架定义威胁模型,并提出一种名为"带掩码的拉格朗日编码"(LCM)的新方案,该方案将服务器分组并融合编码与掩码技术。通过调整每组服务器数量,LCM可实现上行与下行通信负载的权衡。此外,我们分别推导出上下行通信负载的下界,并证明LCM能达到最优上行通信负载,该负载与合谋客户端数量无关。