Outsourcing deep neural networks (DNNs) inference tasks to an untrusted cloud raises data privacy and integrity concerns. While there are many techniques to ensure privacy and integrity for polynomial-based computations, DNNs involve non-polynomial computations. To address these challenges, several privacy-preserving and verifiable inference techniques have been proposed based on replacing the non-polynomial activation functions such as the rectified linear unit (ReLU) function with polynomial activation functions. Such techniques usually require polynomials with integer coefficients or polynomials over finite fields. Motivated by such requirements, several works proposed replacing the ReLU function with the square function. In this work, we empirically show that the square function is not the best degree-2 polynomial that can replace the ReLU function even when restricting the polynomials to have integer coefficients. We instead propose a degree-2 polynomial activation function with a first order term and empirically show that it can lead to much better models. Our experiments on the CIFAR and Tiny ImageNet datasets on various architectures such as VGG-16 show that our proposed function improves the test accuracy by up to 10.4% compared to the square function.

翻译：将深度神经网络（DNN）推理任务外包给不可信云服务器会引发数据隐私与完整性风险。尽管现有多种技术可确保多项式计算场景下的隐私与完整性，但DNN涉及非多项式运算。为应对这些挑战，现有技术通过用多项式激活函数替代修正线性单元（ReLU）等非多项式激活函数，提出了多种隐私保护与可验证推理方案。此类技术通常要求采用整数系数多项式或有限域多项式。受此需求驱动，多项研究提出用平方函数替代ReLU函数。本研究通过实验表明，即便限制多项式具有整数系数，平方函数也并非替代ReLU函数的最佳二次多项式。我们提出一种含一阶项的二次多项式激活函数，实验证明该函数可构建显著更优的模型。在CIFAR和Tiny ImageNet数据集上，基于VGG-16等多种架构的实验表明，相较于平方函数，我们所提函数将测试准确率最高提升10.4%。