Typical machine learning approaches require centralized data for model training, which may not be possible where restrictions on data sharing are in place due to, for instance, privacy and gradient protection. The recently proposed Federated Learning (FL) framework allows learning a shared model collaboratively without data being centralized or shared among data owners. However, we show in this paper that the generalization ability of the joint model is poor on Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when the Federated Averaging (FedAvg) strategy is used due to the weight divergence phenomenon. Hence, we propose a novel boosting algorithm for FL to address both the generalization and gradient leakage issues, as well as achieve faster convergence in gradient-based optimization. In addition, a secure gradient sharing protocol using Homomorphic Encryption (HE) and Differential Privacy (DP) is introduced to defend against gradient leakage attack and avoid pairwise encryption that is not scalable. We demonstrate the proposed Federated Boosting (FedBoosting) method achieves noticeable improvements in both prediction accuracy and run-time efficiency in a visual text recognition task on public benchmark.

翻译：典型的机器学习方法需要集中数据来进行模型训练，但在数据共享受到限制的场景（例如隐私和梯度保护）中可能无法实现。最近提出的联邦学习框架允许在不集中数据或数据所有者之间共享数据的情况下协作学习共享模型。然而，本文表明，联合模型在非独立同分布数据上的泛化能力较差，特别是当使用联邦平均策略时，由于权重发散现象，这一问题尤为突出。因此，我们提出了一种新颖的联邦学习增强算法，以解决泛化能力和梯度泄露问题，同时在基于梯度的优化中实现更快的收敛。此外，我们引入了一种使用同态加密和差分隐私的安全梯度共享协议，以抵御梯度泄露攻击，并避免不可扩展的成对加密。我们提出的联邦增强方法在公开基准测试的视觉文本识别任务中，在预测准确率和运行效率方面均取得了显著提升。