Fully Homomorphic Encryption (FHE) has the potential to substantially improve privacy and security by enabling computation on encrypted data. This is especially true with deep learning, as today many popular user services are powered by neural networks. One of the major challenges facing wide-scale deployment of FHE-secured neural inference is effectively mapping them to the FHE domain. FHE poses many programming challenges including packing large vectors, handling expensive rotations, and correctly implementing complex strided convolutions. This makes programming FHE inferences prone to poor performance and errors. In this paper we overcome these challenges with Orion, an automated optimizing FHE compiler for neural inference. Orion automatically maps PyTorch-specified networks to FHE, handling common layer types and arbitrary tensor shapes and strides. Moreover, we develop novel optimizations that balance dense FHE vector packing, efficient rotations, and minimize operations to improve performance. We have implemented Orion, which will be open sourced, and evaluated it on common benchmarks used by the FHE deep learning community. We compare Orion to multiple state-of-the-art solutions and report iso-accuracy speedups ranging from 2.7$\times$ to 20.5$\times$.

翻译：全同态加密通过支持对密文数据直接计算，具有显著提升隐私与安全性的潜力。这一特性在深度学习领域尤为重要——当前众多主流用户服务均由神经网络驱动。然而，将神经网络推理安全部署到全同态加密域面临多项编程挑战：包括大型向量的打包、高开销旋转操作的处理，以及复杂步长卷积的正确实现等。这使得全同态加密推理程序容易产生性能低下和逻辑错误。本文通过提出Orion编译器解决上述难题，这是一种面向神经网络推理的自动化优化全同态加密编译器。Orion可自动将PyTorch定义的网络映射至全同态加密域，支持常见层类型及任意张量形状与步长。此外，我们开发了新型优化策略，在稠密全同态加密向量打包、高效旋转操作与最小化运算量之间实现平衡，从而提升性能。我们已实现Orion（即将开源），并在全同态加密深度学习社区常用的基准测试上进行评估。将Orion与多种现有最优解决方案对比，在保持精度不变的前提下，实现了2.7倍至20.5倍的加速比。