We present RHODE, a novel system that enables privacy-preserving training of and prediction on Recurrent Neural Networks (RNNs) in a cross-silo federated learning setting by relying on multiparty homomorphic encryption. RHODE preserves the confidentiality of the training data, the model, and the prediction data; and it mitigates federated learning attacks that target the gradients under a passive-adversary threat model. We propose a packing scheme, multi-dimensional packing, for a better utilization of Single Instruction, Multiple Data (SIMD) operations under encryption. With multi-dimensional packing, RHODE enables the efficient processing, in parallel, of a batch of samples. To avoid the exploding gradients problem, RHODE provides several clipping approximations for performing gradient clipping under encryption. We experimentally show that the model performance with RHODE remains similar to non-secure solutions both for homogeneous and heterogeneous data distribution among the data holders. Our experimental evaluation shows that RHODE scales linearly with the number of data holders and the number of timesteps, sub-linearly and sub-quadratically with the number of features and the number of hidden units of RNNs, respectively. To the best of our knowledge, RHODE is the first system that provides the building blocks for the training of RNNs and its variants, under encryption in a federated learning setting.

翻译：本文提出RHODE，一种新颖的系统，通过依赖多方同态加密，在跨孤岛联邦学习环境中实现循环神经网络(RNN)的隐私保护训练与预测。RHODE保障训练数据、模型及预测数据的机密性，并在被动 adversary威胁模型下减轻针对梯度的联邦学习攻击。我们提出一种打包方案——多维打包，以更高效地利用加密状态下的单指令多数据流(SIMD)操作。通过多维打包，RHODE能够并行高效处理批量样本。为避免梯度爆炸问题，RHODE提供了多种裁剪近似方法，用于在加密环境下执行梯度裁剪。实验表明，无论数据持有方之间的数据分布为同质或异质，RHODE的模型性能均与非安全解决方案保持相似。我们的实验评估显示，RHODE的扩展性关于数据持有方数量和时间步长呈线性，关于RNN特征数量呈次线性，关于隐藏单元数量呈次二次性。据我们所知，RHODE是首个在联邦学习加密环境下提供RNN及其变体训练基础组件的系统。