The commercialization of large language models (LLMs) has led to the common practice of high-level API-only access to proprietary models. In this work, we show that even with a conservative assumption about the model architecture, it is possible to learn a surprisingly large amount of non-public information about an API-protected LLM from a relatively small number of API queries (e.g., costing under $1,000 for OpenAI's gpt-3.5-turbo). Our findings are centered on one key observation: most modern LLMs suffer from a softmax bottleneck, which restricts the model outputs to a linear subspace of the full output space. We show that this lends itself to a model image or a model signature which unlocks several capabilities with affordable cost: efficiently discovering the LLM's hidden size, obtaining full-vocabulary outputs, detecting and disambiguating different model updates, identifying the source LLM given a single full LLM output, and even estimating the output layer parameters. Our empirical investigations show the effectiveness of our methods, which allow us to estimate the embedding size of OpenAI's gpt-3.5-turbo to be about 4,096. Lastly, we discuss ways that LLM providers can guard against these attacks, as well as how these capabilities can be viewed as a feature (rather than a bug) by allowing for greater transparency and accountability.

翻译：大型语言模型（LLM）的商业化导致了一种常见做法：仅通过高级API访问专有模型。在这项工作中，我们表明，即使对模型架构做出保守假设，也能通过相对较少的API查询（例如对OpenAI的gpt-3.5-turbo成本低于1000美元）了解到关于API保护LLM的大量非公开信息。我们的发现基于一个关键观察：大多数现代LLM存在softmax瓶颈，这限制了模型输出到完整输出空间的线性子空间。我们证明这可以形成模型映像或模型签名，从而以可承受的成本实现多种能力：高效发现LLM的隐藏层大小、获取全词汇表输出、检测和区分不同的模型更新、根据单个完整LLM输出识别源LLM，甚至估计输出层参数。我们的实证研究显示了方法的有效性，使我们能够估计OpenAI的gpt-3.5-turbo的嵌入大小约为4096。最后，我们讨论了LLM提供商防御这些攻击的方法，以及如何将这些能力视为增强透明度和可问责性的特性（而非缺陷）。