Privacy-Preserving machine learning (PPML) can help us train and deploy models that utilize private information. In particular, on-device Machine Learning allows us to completely avoid sharing information with a third-party server during inference. However, on-device models are typically less accurate when compared to the server counterparts due to the fact that (1) they typically only rely on a small set of on-device features and (2) they need to be small enough to run efficiently on end-user devices. Split Learning (SL) is a promising approach that can overcome these limitations. In SL, a large machine learning model is divided into two parts, with the bigger part residing on the server-side and a smaller part executing on-device, aiming to incorporate the private features. However, end-to-end training of such models requires exchanging gradients at the cut layer, which might encode private features or labels. In this paper, we provide insights into potential privacy risks associated with SL and introduce a novel attack method, EXACT, to reconstruct private information. Furthermore, we also investigate the effectiveness of various mitigation strategies. Our results indicate that the gradients significantly improve the attacker's effectiveness in all three datasets reaching almost 100% reconstruction accuracy for some features. However, a small amount of differential privacy (DP) is quite effective in mitigating this risk without causing significant training degradation.

翻译：隐私保护机器学习能够帮助我们训练和部署利用私有信息的模型。特别是设备端机器学习，可以在推理过程中完全避免与第三方服务器共享信息。然而，由于以下两个原因，设备端模型的准确性通常低于服务器端模型：（1）它们通常仅依赖少量设备端特征；（2）它们需要足够小以便在终端用户设备上高效运行。分割学习是一种能够克服这些限制的前沿方法。在分割学习中，大型机器学习模型被分为两部分：较大部分部署在服务器端，较小部分在设备端执行，旨在整合私有特征。然而，此类模型的端到端训练需要在切割层交换梯度，这可能编码私有特征或标签。本文揭示了分割学习潜在隐私风险的内幕，并提出了一种名为EXACT的新型攻击方法以重构私有信息。此外，我们还研究了各种防御策略的有效性。实验结果表明，梯度在所有三个数据集中显著提升了攻击者的效能，某些特征的恢复准确率几乎达到100%。但少量差分隐私在不会造成明显训练退化的前提下，能有效缓解此类风险。