Large language models (LLMs) can produce long, coherent passages of text, suggesting that LLMs, although trained on next-word prediction, must represent the latent structure that characterizes a document. Prior work has found that internal representations of LLMs encode one aspect of latent structure, namely syntax; here we investigate a complementary aspect, namely the document's topic structure. We motivate the hypothesis that LLMs capture topic structure by connecting LLM optimization to implicit Bayesian inference. De Finetti's theorem shows that exchangeable probability distributions can be represented as a mixture with respect to a latent generating distribution. Although text is not exchangeable at the level of syntax, exchangeability is a reasonable starting assumption for topic structure. We thus hypothesize that predicting the next token in text will lead LLMs to recover latent topic distributions. We examine this hypothesis using Latent Dirichlet Allocation (LDA), an exchangeable probabilistic topic model, as a target, and we show that the representations formed by LLMs encode both the topics used to generate synthetic data and those used to explain natural corpus data.

翻译：大语言模型（LLMs）能够生成连贯的长篇文本，这表明尽管LLMs以预测下一词为训练目标，但它们必然表征了文本的潜在结构特征。先前研究发现LLMs的内部表征编码了潜在结构的一个方面——句法；本文则研究互补的另一个方面，即文本的主题结构。我们通过将LLM优化与隐式贝叶斯推理相联系，提出LLMs捕捉主题结构的假设。德菲内蒂定理表明，可交换概率分布可表示为关于潜在生成分布的混合分布。尽管文本在句法层面不具备可交换性，但可交换性作为主题结构的合理初始假设是成立的。因此我们假设，预测文本中下一词将促使LLMs恢复潜在主题分布。我们以可交换概率主题模型——潜在狄利克雷分配（LDA）为目标检验该假设，结果表明LLMs形成的表征既编码了用于生成合成数据的主题，也编码了用于解释自然语料库数据的主题。