Secure multi-party computation (MPC) techniques can be used to provide data privacy when users query deep neural network (DNN) models hosted on a public cloud. State-of-the-art MPC techniques can be directly leveraged for DNN models that use simple activation functions such as ReLU. However, these techniques are ineffective and/or inefficient for the complex and highly non-linear activation functions used in cutting-edge DNN models. We present Compact, which produces piece-wise polynomial approximations of complex AFs to enable their efficient use with state-of-the-art MPC techniques. Compact neither requires nor imposes any restriction on model training and results in near-identical model accuracy. To achieve this, we design Compact with input density awareness and use an application-specific simulated annealing type optimization to generate computationally more efficient approximations of complex AFs. We extensively evaluate Compact on four different machine-learning tasks with DNN architectures that use popular complex AFs silu, gelu, and mish. Our experimental results show that Compact incurs negligible accuracy loss while being 2x-5x computationally more efficient than state-of-the-art approaches for DNN models with large number of hidden layers. Our work accelerates easy adoption of MPC techniques to provide user data privacy even when the queried DNN models consist of a number of hidden layers and trained over complex AFs.

翻译：安全多方计算（MPC）技术可用于保护用户在公有云上查询深度神经网络（DNN）模型时的数据隐私。对于使用ReLU等简单激活函数的DNN模型，可直接利用最先进的MPC技术。然而，对于前沿DNN模型中使用的复杂且高度非线性的激活函数，这些技术效果不佳或效率低下。我们提出Compact方法，该方法通过生成复杂激活函数的分段多项式逼近，使其能够高效兼容最先进的MPC技术。Compact既不要求也不施加任何模型训练限制，且能保持近乎一致的模型精度。为实现这一目标，我们设计了具备输入密度感知能力的Compact，并采用面向特定应用场景的模拟退火型优化方法，生成计算效率更高的复杂激活函数逼近形式。我们在四个采用silu、gelu和mish等流行复杂激活函数的DNN架构机器学习任务上对Compact进行了全面评估。实验结果表明，对于具有大量隐藏层的DNN模型，Compact在计算效率上较现有最优方案提升2-5倍，且精度损失可忽略不计。我们的工作加速了MPC技术的应用普及，即使被查询的DNN模型包含多个隐藏层且基于复杂激活函数训练，仍能保障用户数据隐私。