This paper presents Flash, an optimized private inference (PI) hybrid protocol utilizing both homomorphic encryption (HE) and secure two-party computation (2PC), which can reduce the end-to-end PI latency for deep CNN models less than 1 minute with CPU. To this end, first, Flash proposes a low-latency convolution algorithm built upon a fast slot rotation operation and a novel data encoding scheme, which results in 4-94x performance gain over the state-of-the-art. Second, to minimize the communication cost introduced by the standard nonlinear activation function ReLU, Flash replaces the entire ReLUs with the polynomial $x^2+x$ and trains deep CNN models with the new activation function. The trained models improve the inference accuracy for CIFAR-10/100 and TinyImageNet by 16% on average (up to 40% for ResNet-32) compared to prior art. Last, Flash proposes an efficient 2PC-based $x^2+x$ evaluation protocol that does not require any offline communication and that reduces the total communication cost to process the activation layer by 84-196x over the state-of-the-art. As a result, the end-to-end PI latency of Flash implemented on CPU is 0.02 minute for CIFAR-100 and 0.57 minute for TinyImageNet classification, while the total data communication is 0.07GB for CIFAR-100 and 0.22GB for TinyImageNet. Flash improves the state-of-the-art PI by 16-45x in latency and 84-196x in communication cost. Moreover, even for ImageNet, Flash can deliver the latency less than 1 minute on CPU with the total communication less than 1GB.

翻译：本文提出Flash，一种优化的私有推理混合协议，该协议结合同态加密与安全两方计算，可将深度CNN模型的端到端私有推理延迟降至CPU环境下1分钟以内。为此，首先，Flash提出基于快速轮转操作与新型数据编码方案的低延迟卷积算法，相较现有最先进方法实现4-94倍性能提升。其次，为最小化标准非线性激活函数ReLU带来的通信开销，Flash使用多项式$x^2+x$替换全部ReLU，并以新激活函数训练深度CNN模型。相较于现有技术，训练所得模型在CIFAR-10/100和TinyImageNet上的推理准确率平均提升16%（ResNet-32最高提升40%）。最后，Flash提出一种无需离线通信的高效两方计算$x^2+x$评估协议，将处理激活层的总通信成本较现有最先进方法降低84-196倍。据此，Flash在CPU上实现的端到端私有推理延迟为CIFAR-100分类0.02分钟、TinyImageNet分类0.57分钟，总数据通信量分别为0.07GB与0.22GB。Flash在延迟与通信成本上分别较现有最先进私有推理方案提升16-45倍与84-196倍。此外，即便对ImageNet数据集，Flash在CPU上的延迟仍可控制在1分钟以内，总通信量低于1GB。