Secure multi-party computation (MPC) techniques can be used to provide data privacy when users query deep neural network (DNN) models hosted on a public cloud. State-of-the-art MPC techniques can be directly leveraged for DNN models that use simple activation functions (AFs) such as ReLU. However, DNN model architectures designed for cutting-edge applications often use complex and highly non-linear AFs. Designing efficient MPC techniques for such complex AFs is an open problem. Towards this, we propose Compact, which produces piece-wise polynomial approximations of complex AFs to enable their efficient use with state-of-the-art MPC techniques. Compact neither requires nor imposes any restriction on model training and results in near-identical model accuracy. We extensively evaluate Compact on four different machine-learning tasks with DNN architectures that use popular complex AFs SiLU, GeLU, and Mish. Our experimental results show that Compact incurs negligible accuracy loss compared to DNN-specific approaches for handling complex non-linear AFs. We also incorporate Compact in two state-of-the-art MPC libraries for privacy-preserving inference and demonstrate that Compact provides 2x-5x speedup in computation compared to the state-of-the-art approximation approach for non-linear functions -- while providing similar or better accuracy for DNN models with large number of hidden layers

翻译：安全多方计算（MPC）技术可用于在用户查询部署于公共云上的深度神经网络（DNN）模型时提供数据隐私保护。现有最先进的MPC技术可直接应用于使用简单激活函数（如ReLU）的DNN模型。然而，面向前沿应用设计的DNN模型架构通常采用复杂且高度非线性的激活函数。针对此类复杂激活函数设计高效的MPC技术仍是一个开放性问题。为此，我们提出Compact方法，该方法通过生成复杂激活函数的分段多项式近似，使其能够高效地应用于最先进的MPC技术。Compact既不要求也不对模型训练施加任何限制，且能保持近乎相同的模型精度。我们在四种不同机器学习任务中，对采用流行复杂激活函数SiLU、GeLU和Mish的DNN架构进行了全面评估。实验结果表明，与处理复杂非线性激活函数的DNN专用方法相比，Compact的精度损失可忽略不计。此外，我们将Compact集成至两种用于隐私保护推理的最先进MPC库中，实验证明：与当前最先进的非线性函数近似方法相比，Compact在计算速度上可实现2倍至5倍的提升——同时为具有大量隐藏层的DNN模型提供相当或更优的精度。