In this work, we present a novel matrix-encoding method that is particularly convenient for neural networks to make predictions in a privacy-preserving manner using homomorphic encryption. Based on this encoding method, we implement a convolutional neural network for handwritten image classification over encryption. For two matrices $A$ and $B$ to perform homomorphic multiplication, the main idea behind it, in a simple version, is to encrypt matrix $A$ and the transpose of matrix $B$ into two ciphertexts respectively. With additional operations, the homomorphic matrix multiplication can be calculated over encrypted matrices efficiently. For the convolution operation, we in advance span each convolution kernel to a matrix space of the same size as the input image so as to generate several ciphertexts, each of which is later used together with the ciphertext encrypting input images for calculating some of the final convolution results. We accumulate all these intermediate results and thus complete the convolution operation. In a public cloud with 40 vCPUs, our convolutional neural network implementation on the MNIST testing dataset takes $\sim$ 287 seconds to compute ten likelihoods of 32 encrypted images of size $28 \times 28$ simultaneously. The data owner only needs to upload one ciphertext ($\sim 19.8$ MB) encrypting these 32 images to the public cloud.

翻译：本文提出一种新颖的矩阵编码方法，该方法特别适用于神经网络基于同态加密进行隐私保护的预测任务。基于此编码方法，我们实现了面向加密手写图像分类的卷积神经网络。对于需要进行同态乘法的两个矩阵$A$和$B$，其核心思想（简化版本）是将矩阵$A$与矩阵$B$的转置分别加密为两个密文。通过附加操作，可在加密矩阵上高效计算同态矩阵乘法。针对卷积运算，我们预先将每个卷积核展开至与输入图像尺寸相同的矩阵空间，从而生成若干密文；每个生成密文随后将与加密输入图像的密文结合，用于计算部分最终卷积结果。通过累加所有这些中间结果，即可完成卷积运算。在配备40个虚拟CPU的公有云环境中，我们的卷积神经网络在MNIST测试集上对32幅尺寸为$28 \times 28$的加密图像同时计算十类似然值耗时约287秒。数据所有者仅需向公有云上传单个密文（约19.8 MB）即可完成这32幅图像的加密传输。