With ChatGPT as a representative, tons of companies have began to provide services based on large Transformers models. However, using such a service inevitably leak users' prompts to the model provider. Previous studies have studied secure inference for Transformer models using secure multiparty computation (MPC), where model parameters and clients' prompts are kept secret. Despite this, these frameworks are still limited in terms of model performance, efficiency, and deployment. To address these limitations, we propose framework PUMA to enable fast and secure Transformer model inference. Our framework designs high quality approximations for expensive functions, such as GeLU and Softmax, which significantly reduce the cost of secure inference while preserving the model performance. Additionally, we design secure Embedding and LayerNorm procedures that faithfully implement the desired functionality without undermining the Transformer architecture. PUMA is about 2x faster than the state-of-the-art MPC framework MPCFORMER(ICLR 2023) and has similar accuracy as plaintext models without fine-tuning (which the previous works failed to achieve). One more thing, PUMA can evaluate LLaMA-7B in around 5 minutes to generate 1 token. To our best knowledge, this is the first time that a model with such a parameter size is able to be evaluated under MPC. PUMA has been open-sourced in the Github repository of SecretFlow-SPU.

翻译：以ChatGPT为代表的众多企业已开始提供基于大型Transformer模型的服务。然而，使用此类服务时，用户的提示词不可避免地会泄露给模型提供方。已有研究利用安全多方计算（MPC）实现了Transformer模型的安全推理，确保模型参数与客户端提示词的机密性。尽管如此，现有框架在模型性能、效率及部署方面仍存在局限。为突破这些限制，我们提出PUMA框架，旨在实现快速且安全的Transformer模型推理。该框架为GeLU和Softmax等昂贵函数设计了高质量近似方案，在保持模型性能的同时显著降低安全推理成本。此外，我们设计了安全的Embedding和LayerNorm流程，在不破坏Transformer架构的前提下忠实实现所需功能。PUMA的速度约比当前最优MPC框架MPCFORMER（ICLR 2023）快2倍，且在无需微调的情况下即可达到与明文模型相当的精度（此项为先前工作未能实现的目标）。更值得关注的是，PUMA可在约5分钟内完成LLaMA-7B模型的一个token生成推理。据我们所知，这是首次在MPC环境下实现如此参数量级模型的评估。PUMA已在SecretFlow-SPU的GitHub仓库中开源。