With ChatGPT as a representative, tons of companies have began to provide services based on large Transformers models. However, using such a service inevitably leak users' prompts to the model provider. Previous studies have studied secure inference for Transformer models using secure multiparty computation (MPC), where model parameters and clients' prompts are kept secret. Despite this, these frameworks are still limited in terms of model performance, efficiency, and deployment. To address these limitations, we propose framework PUMA to enable fast and secure Transformer model inference. Our framework designs high quality approximations for expensive functions, such as GeLU and Softmax, which significantly reduce the cost of secure inference while preserving the model performance. Additionally, we design secure Embedding and LayerNorm procedures that faithfully implement the desired functionality without undermining the Transformer architecture. PUMA is about 2x faster than the state-of-the-art MPC framework MPCFORMER(ICLR 2023) and has similar accuracy as plaintext models without fine-tuning (which the previous works failed to achieve). One more thing, PUMA can evaluate LLaMA-7B in around 5 minutes to generate 1 token. To our best knowledge, this is the first time that a model with such a parameter size is able to be evaluated under MPC. PUMA has been open-sourced in the Github repository of SecretFlow-SPU.

翻译：以ChatGPT为代表，众多公司已开始提供基于大型Transformer模型的服务。然而，使用此类服务不可避免地会将用户的提示信息泄露给模型提供方。此前研究已探索利用安全多方计算（MPC）对Transformer模型进行安全推理，在保护模型参数和客户端提示隐私的前提下运行。尽管如此，这些框架在模型性能、效率及部署方面仍存在局限。为解决上述问题，我们提出PUMA框架，旨在实现快速且安全的Transformer模型推理。本框架为GeLU、Softmax等计算密集型函数设计了高质量近似方法，在显著降低安全推理成本的同时保持模型性能。此外，我们设计了安全的嵌入（Embedding）与层归一化（LayerNorm）流程，在不破坏Transformer架构的前提下忠实实现预期功能。PUMA较最先进的MPC框架MPCFORMER（发表于ICLR 2023）提速约2倍，且无需微调即可达到与明文模型相当的精度（此前研究未能实现该目标）。更值得注意的是，PUMA可在约5分钟内完成LLaMA-7B模型一个token的生成。据我们所知，这是首次能够在MPC环境下评估如此参数规模的模型。PUMA已作为开源项目发布于SecretFlow-SPU的GitHub仓库。