In two-party machine learning prediction services, the client's goal is to query a remote server's trained machine learning model to perform neural network inference in some application domain. However, sensitive information can be obtained during this process by either the client or the server, leading to potential collection, unauthorized secondary use, and inappropriate access to personal information. These security concerns have given rise to Private Inference (PI), in which both the client's personal data and the server's trained model are kept confidential. State-of-the-art PI protocols consist of a pre-processing or offline phase and an online phase that combine several cryptographic primitives: Homomorphic Encryption (HE), Secret Sharing (SS), Garbled Circuits (GC), and Oblivious Transfer (OT). Despite the need and recent performance improvements, PI remains largely arcane today and is too slow for practical use. This paper addresses PI's shortcomings with a detailed characterization of a standard high-performance protocol to build foundational knowledge and intuition in the systems community. Our characterization pinpoints all sources of inefficiency -- compute, communication, and storage. In contrast to prior work, we consider inference request arrival rates rather than studying individual inferences in isolation and we find that the pre-processing phase cannot be ignored and is often incurred online as there is insufficient downtime to hide pre-compute latency. Finally, we leverage insights from our characterization and propose three optimizations to address the storage (Client-Garbler), computation (layer-parallel HE), and communication (wireless slot allocation) overheads. Compared to the state-of-the-art PI protocol, these optimizations provide a total PI speedup of 1.8$\times$ with the ability to sustain inference requests up to a 2.24$\times$ greater rate.

翻译：在两方机器学习预测服务中，客户端的目标是查询远程服务器上已训练的机器学习模型，以在特定应用领域执行神经网络推理。然而，在这一过程中，客户端或服务器可能获取敏感信息，导致潜在的个人信息收集、未经授权的二次使用及不当访问。这些安全问题催生了私有推理（PI），即要求客户端的个人数据和服务器的训练模型均保持机密。现有最先进的PI协议由预处理（离线）阶段和在线阶段组成，结合了多种密码学原语：同态加密（HE）、秘密共享（SS）、混淆电路（GC）和不经意传输（OT）。尽管有实际需求且近期性能有所提升，但PI目前仍高度晦涩，且因速度过慢而难以实际应用。本文通过详细特征化一个标准的高性能协议来弥补PI的不足，为系统社区建立基础知识和直观理解。我们的特征化精准定位了所有低效率来源——计算、通信和存储。与以往研究不同，我们考虑推理请求的到达率而非单独研究单个推理，并发现预处理阶段不可忽视，且常常在线进行，因为缺乏足够空闲时间隐藏预计算延迟。最后，我们利用特征化中的洞见，提出三项优化以解决存储（客户端-混淆方）、计算（层并行HE）和通信（无线时隙分配）开销。与现有最先进的PI协议相比，这些优化提供了总计1.8倍的PI加速，并能维持高达2.24倍推理请求的到达率。