In this work, we present a novel matrix-encoding method that is particularly convenient for neural networks to make predictions in a privacy-preserving manner using homomorphic encryption. Based on this encoding method, we implement a convolutional neural network for handwritten image classification over encryption. For two matrices $A$ and $B$ to perform homomorphic multiplication, the main idea behind it, in a simple version, is to encrypt matrix $A$ and the transpose of matrix $B$ into two ciphertexts respectively. With additional operations, the homomorphic matrix multiplication can be calculated over encrypted matrices efficiently. For the convolution operation, we in advance span each convolution kernel to a matrix space of the same size as the input image so as to generate several ciphertexts, each of which is later used together with the ciphertext encrypting input images for calculating some of the final convolution results. We accumulate all these intermediate results and thus complete the convolution operation. In a public cloud with 40 vCPUs, our convolutional neural network implementation on the MNIST testing dataset takes $\sim$ 287 seconds to compute ten likelihoods of 32 encrypted images of size $28 \times 28$ simultaneously. The data owner only needs to upload one ciphertext ($\sim 19.8$ MB) encrypting these 32 images to the public cloud.

翻译：在这项工作中，我们提出了一种新型的矩阵编码方法，特别适用于神经网络使用同态加密以隐私保护的方式进行预测。基于这种编码方法，我们实现了一个卷积神经网络，用于对加密的手写图像进行分类。对于要执行同态乘法的两个矩阵$A$和$B$，其背后的主要思想（简单版本）就是将矩阵$A$和矩阵$B$的转置分别加密成两个密文。通过其他操作，可以高效地在加密矩阵上计算同态矩阵乘法。对于卷积操作，我们提前将每个卷积核扩展为与输入图像相同尺寸的矩阵空间，以便生成多个密文，其中每个密文都与加密输入图像一起用于计算最终的某些卷积结果。我们累加所有这些中间结果，从而完成卷积操作。在具有40个vCPUs的公共云中，我们的卷积神经网络实现在MNIST测试数据集上需要约387秒才能同时计算32个大小为$28 \times 28$的加密图像的十个可能性。数据所有者只需要将一个加密这32个图像的密文（约19.8 MB）上传到公共云中。