Deep learning inference brings together the data and the Convolutional Neural Network (CNN). This is problematic in case the user wants to preserve the privacy of the data and the service provider does not want to reveal the weights of his CNN. Secure Inference allows the two parties to engage in a protocol that preserves their respective privacy concerns, while revealing only the inference result to the user. This is known as Multi-Party Computation (MPC). A major bottleneck of MPC algorithms is communication, as the parties must send data back and forth. The linear component of a CNN (i.e. convolutions) can be done efficiently with minimal communication, but the non-linear part (i.e., ReLU) requires the bulk of communication bandwidth. We propose two ways to accelerate Secure Inference. The first is based on the observation that the ReLU outcome of many convolutions is highly correlated. Therefore, we replace the per pixel ReLU operation by a ReLU operation per patch. Each layer in the network will benefit from a patch of a different size and we devise an algorithm to choose the optimal set of patch sizes through a novel reduction of the problem to a knapsack problem. The second way to accelerate Secure Inference is based on cutting the number of bit comparisons required for a secure ReLU operation. We demonstrate the cumulative effect of these tools in the semi-honest secure 3-party setting for four problems: Classifying ImageNet using ResNet50 backbone, classifying CIFAR100 using ResNet18 backbone, semantic segmentation of ADE20K using MobileNetV2 backbone and semantic segmentation of Pascal VOC 2012 using ResNet50 backbone. Our source code is publicly available: $\href{https://github.com/yg320/secure_inference}{\text{https://github.com/yg320/secure_inference}}$

翻译：深度学习推理过程同时涉及用户数据与卷积神经网络（CNN）的交互。当用户希望保护数据隐私，而服务提供商不希望泄露CNN权重时，这一过程便存在安全隐患。安全推理机制允许双方通过协议在保护各自隐私关切的同时，仅向用户公开推理结果，这被称为多方计算（MPC）。MPC算法的主要瓶颈在于通信开销——双方需频繁交换数据。CNN的线性组件（如卷积运算）可通过最小化通信高效实现，但非线性部分（如ReLU函数）占据了大部分通信带宽。我们提出两种加速安全推理的方法：第一种基于“多个卷积的ReLU输出具有高度相关性”的观察结果。具体而言，我们将逐像素的ReLU操作替换为逐补丁的ReLU操作，网络各层可采用不同尺寸的补丁，并通过将问题创新性地转化为背包优化问题来设计算法，筛选出最优补丁尺寸组合。第二种加速方法通过减少安全ReLU操作所需的比特比较次数来实现。我们在半诚实安全三方计算场景下，针对四项任务验证了上述工具的综合效果：使用ResNet50骨干网络对ImageNet进行分类，使用ResNet18骨干网络对CIFAR100进行分类，使用MobileNetV2骨干网络对ADE20K进行语义分割，以及使用ResNet50骨干网络对Pascal VOC 2012进行语义分割。我们的源代码已公开：$\href{https://github.com/yg320/secure_inference}{\text{https://github.com/yg320/secure_inference}}$