We introduce a novel strategy to train randomised predictors in federated learning, where each node of the network aims at preserving its privacy by releasing a local predictor but keeping secret its training dataset with respect to the other nodes. We then build a global randomised predictor which inherits the properties of the local private predictors in the sense of a PAC-Bayesian generalisation bound. We consider the synchronous case where all nodes share the same training objective (derived from a generalisation bound), and the asynchronous case where each node may have its own personalised training objective. We show through a series of numerical experiments that our approach achieves a comparable predictive performance to that of the batch approach where all datasets are shared across nodes. Moreover the predictors are supported by numerically nonvacuous generalisation bounds while preserving privacy for each node. We explicitly compute the increment on predictive performance and generalisation bounds between batch and federated settings, highlighting the price to pay to preserve privacy.

翻译：我们提出了一种在联邦学习中训练随机预测器的新策略，其中网络中的每个节点通过发布本地预测器来保护其隐私，同时对其训练数据集向其他节点保密。随后，我们构建了一个全局随机预测器，该预测器在PAC-贝叶斯泛化边界的意义上继承了本地私有预测器的性质。我们考虑了同步情况（所有节点共享相同的训练目标，该目标源自泛化边界）和异步情况（每个节点可能具有其个性化的训练目标）。通过一系列数值实验，我们表明该方法在预测性能上与所有节点共享数据集的批量方法相当。此外，这些预测器由数值上非平凡的泛化边界支撑，同时保护了每个节点的隐私。我们明确计算了批量设置与联邦设置之间预测性能和泛化边界的增量，揭示了为保护隐私所需付出的代价。