Cloud deep learning platforms provide cost-effective deep neural network (DNN) training for customers who lack computation resources. However, cloud systems are often untrustworthy and vulnerable to attackers, leading to growing concerns about model privacy. Recently, researchers have sought to protect data privacy in deep learning by leveraging CPU trusted execution environments (TEEs), which minimize the use of cryptography, but existing works failed to simultaneously utilize the computational resources of GPUs to assist in training and prevent model leakage. This paper presents Tempo, the first cloud-based deep learning system that cooperates with TEE and distributed GPUs for efficient DNN training with model confidentiality preserved. To tackle the challenge of preserving privacy while offloading linear algebraic operations from TEE to GPUs for efficient batch computation, we introduce a customized permutation-based obfuscation algorithm to blind both inputs and model parameters. An optimization mechanism that reduces encryption operations is proposed for faster weight updates during backpropagation to speed up training. We implement Tempo and evaluate it with both training and inference for two prevalent DNNs. Empirical results indicate that Tempo outperforms baselines and offers sufficient privacy protection.

翻译：云深度学习平台为缺乏计算资源的客户提供了经济高效的深度神经网络（DNN）训练。然而，云系统通常不可信且易受攻击，导致对模型隐私的担忧日益增加。近年来，研究人员试图通过利用CPU可信执行环境（TEE）来保护深度学习中的数据隐私，这种方法最大限度地减少了密码学的使用，但现有工作未能同时利用GPU的计算资源辅助训练并防止模型泄露。本文提出Tempo，这是首个与TEE和分布式GPU协作的基于云的深度学习系统，可在保持模型机密性的同时实现高效的DNN训练。为解决在将线性代数运算从TEE卸载到GPU以实现高效批量计算时保护隐私的挑战，我们引入了一种定制的基于排列的混淆算法来盲化输入和模型参数。提出了一种减少加密操作的优化机制，用于在反向传播期间更快地更新权重，从而加速训练。我们实现了Tempo，并使用两种主流DNN的训练和推理对其进行了评估。实验结果表明，Tempo优于基线方法，并提供了充分的隐私保护。