As large language models (LLMs) continue to grow in size, fewer users are able to host and run models locally. This has led to increased use of third-party hosting services. However, in this setting, there is a lack of guarantees on the computation performed by the inference provider. For example, a dishonest provider may replace an expensive large model with a cheaper-to-run weaker model and return the results from the weaker model to the user. Existing tools to verify inference typically rely on methods from cryptography such as zero-knowledge proofs (ZKPs), but these add significant computational overhead, and remain infeasible for use for large models. In this work, we develop a new insight -- that given a method for performing private LLM inference, one can obtain forms of verified inference at marginal extra cost. Specifically, we propose two new protocols which leverage privacy-preserving LLM inference in order to provide guarantees over the inference that was carried out. Our approaches are cheap, requiring the addition of a few extra tokens of computation, and have little to no downstream impact. As the fastest privacy-preserving inference methods are typically faster than ZK methods, the proposed protocols also improve verification runtime. Our work provides novel insights into the connections between privacy and verifiability in LLM inference.

翻译：随着大型语言模型（LLMs）规模持续增长，能够在本地部署和运行模型的用户日益减少。这导致第三方托管服务的应用逐渐增多。然而，在此类场景中，推理服务提供商所执行的计算缺乏可靠性保证。例如，不诚实的提供商可能将昂贵的大型模型替换为运行成本较低的弱化模型，并将弱化模型的输出结果返回给用户。现有的推理验证工具通常依赖零知识证明（ZKPs）等密码学方法，但这些方法会带来显著的计算开销，对于大型模型而言仍不具备可行性。本研究提出创新观点：若存在实现隐私保护型LLM推理的方法，则能以极低的边际成本获得多种形式的可验证推理。具体而言，我们提出两种新型协议，通过利用隐私保护型LLM推理机制为已执行的推理过程提供可靠性保证。我们的方法成本低廉——仅需增加少量额外词元计算量，且对下游任务影响甚微。由于当前最快的隐私保护推理方法通常快于零知识证明方法，所提出的协议还能提升验证运行效率。本研究为LLM推理中隐私性与可验证性之间的关联机制提供了新的理论洞见。