In this work, we present a novel matrix-encoding method that is particularly convenient for neural networks to make predictions in a privacy-preserving manner using homomorphic encryption. Based on this encoding method, we implement a convolutional neural network for handwritten image classification over encryption. For two matrices $A$ and $B$ to perform homomorphic multiplication, the main idea behind it, in a simple version, is to encrypt matrix $A$ and the transpose of matrix $B$ into two ciphertexts respectively. With additional operations, the homomorphic matrix multiplication can be calculated over encrypted matrices efficiently. For the convolution operation, we in advance span each convolution kernel to a matrix space of the same size as the input image so as to generate several ciphertexts, each of which is later used together with the ciphertext encrypting input images for calculating some of the final convolution results. We accumulate all these intermediate results and thus complete the convolution operation. In a public cloud with 40 vCPUs, our convolutional neural network implementation on the MNIST testing dataset takes $\sim$ 287 seconds to compute ten likelihoods of 32 encrypted images of size $28 \times 28$ simultaneously. The data owner only needs to upload one ciphertext ($\sim 19.8$ MB) encrypting these 32 images to the public cloud.

翻译：本文提出了一种新型矩阵编码方法，特别适用于利用同态加密以隐私保护方式执行神经网络预测。基于该编码方法，我们实现了一个用于加密手写图像分类的卷积神经网络。对于两个矩阵$A$和$B$进行同态乘法运算，其核心思想（简化版本）是将矩阵$A$与矩阵$B$的转置分别加密为两个密文。通过附加操作，可在加密矩阵上高效计算同态矩阵乘法。针对卷积操作，我们预先将每个卷积核扩展至与输入图像相同尺寸的矩阵空间，从而生成多个密文，每个密文后续与加密输入图像的密文结合，用于计算部分最终卷积结果。通过累加所有中间结果完成卷积操作。在配备40个vCPU的公共云环境下，基于MNIST测试数据集实现的卷积神经网络，可同时计算32幅大小为$28 \times 28$的加密图像的十个似然值，耗时约287秒。数据所有者仅需向公共云上传一个加密这32幅图像的密文（约19.8 MB）。