Fully homomorphic encryption (FHE) is a promising cryptographic primitive for realizing private neural network inference (PI) services by allowing a client to fully offload the inference task to a cloud server while keeping the client data oblivious to the server. This work proposes NeuJeans, an FHE-based solution for the PI of deep convolutional neural networks (CNNs). NeuJeans tackles the critical problem of the enormous computational cost for the FHE evaluation of convolutional layers (conv2d), mainly due to the high cost of data reordering and bootstrapping. We first propose an encoding method introducing nested structures inside encoded vectors for FHE, which enables us to develop efficient conv2d algorithms with reduced data reordering costs. However, the new encoding method also introduces additional computations for conversion between encoding methods, which could negate its advantages. We discover that fusing conv2d with bootstrapping eliminates such computations while reducing the cost of bootstrapping. Then, we devise optimized execution flows for various types of conv2d and apply them to end-to-end implementation of CNNs. NeuJeans accelerates the performance of conv2d by up to 5.68 times compared to state-of-the-art FHE-based PI work and performs the PI of a CNN at the scale of ImageNet (ResNet18) within a mere few seconds

翻译：全同态加密（FHE）是一种前景广阔的密码学原语，通过允许客户端将推理任务完全卸载至云服务器同时保持客户端数据对服务器不可见，可实现私有神经网络推理（PI）服务。本文提出NeuJeans——基于FHE的深度卷积神经网络（CNN）私有推理解决方案。NeuJeans着重解决卷积层（conv2d）的FHE评估中因数据重排与自举操作的高昂成本导致的巨大计算开销问题。我们首先提出一种在编码向量内引入嵌套结构的FHE编码方法，该方法能够开发出降低数据重排成本的高效conv2d算法。然而，新编码方法同时引入了编码格式转换的额外计算，可能抵消其优势。我们发现将conv2d与自举操作融合可以消除此类计算并降低自举开销。随后，我们为各类conv2d设计了优化执行流程，并将其应用于CNN的端到端实现。相比现有最优的基于FHE的PI方案，NeuJeans将conv2d性能提升最高达5.68倍，并在ImageNet规模（ResNet18）的CNN上仅需数秒即可完成私有推理。