Fully homomorphic encryption (FHE) is a promising cryptographic primitive for realizing private neural network inference (PI) services by allowing a client to fully offload the inference task to a cloud server while keeping the client data oblivious to the server. This work proposes NeuJeans, an FHE-based solution for the PI of deep convolutional neural networks (CNNs). NeuJeans tackles the critical problem of the enormous computational cost for the FHE evaluation of CNNs. We introduce a novel encoding method called Coefficients-in-Slot (CinS) encoding, which enables multiple convolutions in one HE multiplication without costly slot permutations. We further observe that CinS encoding is obtained by conducting the first several steps of the Discrete Fourier Transform (DFT) on a ciphertext in conventional Slot encoding. This property enables us to save the conversion between CinS and Slot encodings as bootstrapping a ciphertext starts with DFT. Exploiting this, we devise optimized execution flows for various two-dimensional convolution (conv2d) operations and apply them to end-to-end CNN implementations. NeuJeans accelerates the performance of conv2d-activation sequences by up to 5.68 times compared to state-of-the-art FHE-based PI work and performs the PI of a CNN at the scale of ImageNet within a mere few seconds.

翻译：全同态加密（FHE）是一种前景广阔的密码学原语，可用于实现私有神经网络推理（PI）服务，其允许客户端将推理任务完全卸载至云服务器，同时确保服务器无法获知客户端数据。本研究提出NeuJeans，一种基于FHE的深度卷积神经网络（CNN）PI解决方案。NeuJeans旨在解决CNN在FHE评估中计算成本极高的关键问题。我们引入了一种称为“槽内系数”（CinS）编码的新型编码方法，该方法可在单次同态加密乘法中实现多次卷积运算，而无需昂贵的槽置换操作。我们进一步观察到，通过对传统槽编码下的密文执行离散傅里叶变换（DFT）的前若干步骤，即可获得CinS编码。这一特性使得我们能够在自举操作（其以DFT为起始步骤）中节省CinS与槽编码之间的转换开销。基于此，我们为多种二维卷积（conv2d）操作设计了优化执行流程，并将其应用于端到端的CNN实现中。与当前最先进的基于FHE的PI工作相比，NeuJeans将conv2d-激活序列的执行性能提升最高达5.68倍，并能在仅数秒内完成ImageNet规模级别的CNN私有推理。