With ChatGPT as a representative, tons of companies have began to provide services based on large Transformers models. However, using such a service inevitably leak users' prompts to the model provider. Previous studies have studied secure inference for Transformer models using secure multiparty computation (MPC), where model parameters and clients' prompts are kept secret. Despite this, these frameworks are still limited in terms of model performance, efficiency, and deployment. To address these limitations, we propose framework PUMA to enable fast and secure Transformer model inference. Our framework designs high quality approximations for expensive functions such as GeLU and softmax, and significantly reduce the cost of secure inference while preserving the model performance. Additionally, we design secure Embedding and LayerNorm procedures that faithfully implement the desired functionality without undermining the Transformer architecture. PUMA is about $2\times$ faster than the state-of-the-art framework MPCFORMER(ICLR 2023) and has similar accuracy as plaintext models without fine-tuning (which the previous works failed to achieve). PUMA can even evaluate LLaMA-7B in around 5 minutes to generate 1 token. To our best knowledge, this is the first time that a model with such a parameter size is able to be evaluated under MPC. PUMA has been open-sourced in the Github repository of SecretFlow-SPU.

翻译：以ChatGPT为代表，众多公司已开始基于大型Transformer模型提供服务。然而，使用此类服务不可避免地会将用户的提示词泄露给模型提供方。已有研究利用安全多方计算（MPC）探索了Transformer模型的安全推理方案，可在保护模型参数与客户提示词隐私的同时完成计算。尽管如此，现有框架在模型性能、效率与部署方面仍存在局限。为解决这些问题，我们提出PUMA框架，旨在实现快速安全的Transformer模型推理。该框架针对GeLU和softmax等高开销函数设计了高质量近似方法，在保持模型性能的同时显著降低安全推理成本。此外，我们设计了安全Embedding与LayerNorm流程，在忠实实现预期功能的同时不破坏Transformer架构。PUMA的运行速度比当前最先进框架MPCFORMER（ICLR 2023）快约2倍，且无需微调即可达到与明文模型相当的精度（先前工作未能实现这一目标）。PUMA甚至可在约5分钟内完成LLaMA-7B模型单个token的生成。据我们所知，这是首次在MPC环境下实现如此参数规模模型的推理评估。PUMA已在SecretFlow-SPU的Github仓库中开源。