Convolutional neural network (CNN) inference using fully homomorphic encryption (FHE) is a promising private inference (PI) solution due to the capability of FHE that enables offloading the whole computation process to the server while protecting the privacy of sensitive user data. However, prior FHEbased CNN (HCNN) implementations are far from being practical due to the high computational and memory overheads of FHE. To overcome this limitation, we present HyPHEN, a deep HCNN construction that features an efficient FHE convolution algorithm, data packing methods (hybrid packing and image slicing), and FHE-specific optimizations. Such enhancements enable HyPHEN to substantially reduce the memory footprint and the number of expensive homomorphic operations, such as ciphertext rotation and bootstrapping. As a result, HyPHEN brings the latency of HCNN CIFAR-10 inference down to a practical level at 1.40s (ResNet20) and demonstrates HCNN ImageNet inference for the first time at 16.87s (ResNet18).

翻译：基于全同态加密（FHE）的卷积神经网络（CNN）推理是一种有前景的隐私推理（PI）方案，其优势在于FHE能够将整个计算过程外包至服务器端，同时保护用户敏感数据的隐私。然而，受限于FHE高昂的计算与存储开销，现有基于FHE的CNN（HCNN）实现远未达到实用化水平。为突破这一局限，我们提出HyPHEN——一种深度HCNN架构，其核心包括高效的FHE卷积算法、数据打包方法（混合打包与图像切片）以及FHE专用优化技术。这些改进使HyPHEN能够显著降低内存占用，并减少密文旋转与自举等高开销同态运算的次数。实验表明，HyPHEN将HCNN在CIFAR-10上的推理延迟降至实用水平（ResNet20：1.40秒），并在ImageNet上首次实现HCNN推理（ResNet18：16.87秒）。