Privacy-Preserving machine learning (PPML) can help us train and deploy models that utilize private information. In particular, on-device Machine Learning allows us to completely avoid sharing information with a third-party server during inference. However, on-device models are typically less accurate when compared to the server counterparts due to the fact that (1) they typically only rely on a small set of on-device features and (2) they need to be small enough to run efficiently on end-user devices. Split Learning (SL) is a promising approach that can overcome these limitations. In SL, a large machine learning model is divided into two parts, with the bigger part residing on the server-side and a smaller part executing on-device, aiming to incorporate the private features. However, end-to-end training of such models requires exchanging gradients at the cut layer, which might encode private features or labels. In this paper, we provide insights into potential privacy risks associated with SL and introduce a novel attack method, EXACT, to reconstruct private information. Furthermore, we also investigate the effectiveness of various mitigation strategies. Our results indicate that the gradients significantly improve the attacker's effectiveness in all three datasets reaching almost 100% reconstruction accuracy for some features. However, a small amount of differential privacy (DP) is quite effective in mitigating this risk without causing significant training degradation.

翻译：隐私保护机器学习（PPML）可帮助我们在利用私有信息的同时训练和部署模型。特别是，设备端机器学习允许在推理阶段完全避免与第三方服务器共享信息。然而，设备端模型通常比服务器端模型精度更低，原因在于：（1）它们通常仅依赖少量设备端特征；（2）需足够小巧以在终端用户设备上高效运行。拆分学习（SL）是一种能够克服这些限制的有前景方法。在SL中，大型机器学习模型被分为两部分：较大部分驻留在服务器端，较小部分在设备端执行，旨在融合私有特征。然而，此类模型的端到端训练需要在切割层交换梯度，这可能编码了私有特征或标签。本文深入分析了SL中潜在的隐私风险，并提出了一种名为EXACT的新型攻击方法，用于重建私有信息。此外，我们还研究了多种缓解策略的有效性。实验结果表明，梯度显著提升了攻击者在三个数据集上的有效性，部分特征的恢复准确率几乎达到100%。尽管如此，少量差分隐私（DP）可在不导致严重训练性能下降的情况下有效缓解此类风险。