The commercialization of large language models (LLMs) has led to the common practice of high-level API-only access to proprietary models. In this work, we show that even with a conservative assumption about the model architecture, it is possible to learn a surprisingly large amount of non-public information about an API-protected LLM from a relatively small number of API queries (e.g., costing under $1,000 for OpenAI's gpt-3.5-turbo). Our findings are centered on one key observation: most modern LLMs suffer from a softmax bottleneck, which restricts the model outputs to a linear subspace of the full output space. We show that this lends itself to a model image or a model signature which unlocks several capabilities with affordable cost: efficiently discovering the LLM's hidden size, obtaining full-vocabulary outputs, detecting and disambiguating different model updates, identifying the source LLM given a single full LLM output, and even estimating the output layer parameters. Our empirical investigations show the effectiveness of our methods, which allow us to estimate the embedding size of OpenAI's gpt-3.5-turbo to be about 4,096. Lastly, we discuss ways that LLM providers can guard against these attacks, as well as how these capabilities can be viewed as a feature (rather than a bug) by allowing for greater transparency and accountability.

翻译：大语言模型（LLM）的商业化导致仅通过高级API访问专有模型成为普遍做法。在本研究中，我们证明即使对模型架构持保守假设，仍可通过相对少量的API查询（例如，针对OpenAI的gpt-3.5-turbo成本低于1000美元）获知大量关于API保护LLM的非公开信息。我们的发现基于一个关键观察：大多数现代LLM存在softmax瓶颈，这限制了模型输出至完整输出空间的线性子空间。我们表明，这形成了模型映像或模型签名，能以可承受的成本解锁多项能力：高效发现LLM的隐藏维度、获取全词汇输出、检测并区分不同模型更新、根据单个完整LLM输出识别源模型，甚至估算输出层参数。实证研究证明了我们方法的有效性，据此我们估算OpenAI的gpt-3.5-turbo的嵌入维度约为4096。最后，我们讨论了LLM提供商防范此类攻击的途径，以及如何将这些能力视为一种特性（而非缺陷）以促进更高的透明度和问责性。