Private convolutional neural network (CNN) inference based on secure two-party computation (2PC) suffers from high communication and latency overhead, especially from convolution layers. In this paper, we propose UFO, a quantized 2PC inference framework that jointly optimizes the 2PC protocols and quantization algorithm. UFO features a novel 2PC protocol that systematically combines the efficient Winograd convolution algorithm with quantization to improve inference efficiency. However, we observe that naively combining quantization and Winograd convolution faces the following challenges: 1) From the inference perspective, Winograd transformations introduce extensive additions and require frequent bit width conversions to avoid inference overflow, leading to non-negligible communication overhead; 2) From the training perspective, Winograd transformations introduce weight outliers that make quantization-aware training (QAT) difficult, resulting in inferior model accuracy. To address these challenges, we co-optimize both protocol and algorithm. 1) At the protocol level, we propose a series of graph-level optimizations for 2PC inference to minimize the communication. 2) At the algorithm level, we develop a mixed-precision QAT algorithm based on layer sensitivity to optimize model accuracy given communication constraints. To accommodate the outliers, we further introduce a 2PC-friendly bit re-weighting algorithm to increase the representation range without explicitly increasing bit widths. With extensive experiments, UFO demonstrates 11.7x, 3.6x, and 6.3x communication reduction with 1.29%, 1.16%, and 1.29% higher accuracy compared to state-of-the-art frameworks SiRNN, COINN, and CoPriv, respectively.

翻译：基于安全两方计算（2PC）的私有卷积神经网络（CNN）推理存在高通信开销和高延迟问题，卷积层尤为突出。本文提出UFO，一个量化2PC推理框架，通过联合优化2PC协议与量化算法来解决此问题。UFO的核心是一种新颖的2PC协议，该协议系统性地将高效的Winograd卷积算法与量化技术相结合以提升推理效率。然而，我们发现简单地将量化与Winograd卷积结合面临以下挑战：1）从推理角度看，Winograd变换引入大量加法运算，并需要频繁的位宽转换以避免推理溢出，导致不可忽视的通信开销；2）从训练角度看，Winograd变换引发权重异常值，使得量化感知训练（QAT）难以进行，导致模型精度下降。为应对这些挑战，我们进行了协议与算法的协同优化。1）在协议层面，我们提出一系列面向2PC推理的图级优化以最小化通信开销；2）在算法层面，我们开发了一种基于层敏感度的混合精度QAT算法，在给定通信约束下优化模型精度。针对异常值问题，我们进一步引入了一种对2PC友好的比特重加权算法，可在不显式增加位宽的情况下扩大表示范围。大量实验表明，与最先进的框架SiRNN、COINN和CoPriv相比，UFO分别实现了11.7倍、3.6倍和6.3倍的通信量降低，同时精度分别高出1.29%、1.16%和1.29%。