In this paper, we introduce GhostWriteBench, a dataset for LLM authorship attribution. It comprises long-form texts (50K+ words per book) generated by frontier LLMs, and is designed to test generalisation across multiple out-of-distribution (OOD) dimensions, including domain and unseen LLM author. We also propose TRACE -- a novel fingerprinting method that is interpretable and lightweight -- that works for both open- and closed-source models. TRACE creates the fingerprint by capturing token-level transition patterns (e.g., word rank) estimated by another lightweight language model. Experiments on GhostWriteBench demonstrate that TRACE achieves state-of-the-art performance, remains robust in OOD settings, and works well in limited training data scenarios.

翻译：本文介绍了GhostWriteBench——一个用于大语言模型（LLM）作者归因的数据集。该数据集包含由前沿LLM生成的长篇文本（每本书超过5万字），旨在测试跨多个分布外（OOD）维度的泛化能力，包括领域和未见过的LLM作者。我们还提出了TRACE——一种可解释且轻量级的新型指纹提取方法，适用于开源和闭源模型。TRACE通过捕获由另一个轻量级语言模型估计的token级转换模式（例如词排序）来生成指纹。在GhostWriteBench上的实验表明，TRACE实现了最先进的性能，在OOD设置下保持鲁棒性，并在训练数据有限的情况下表现良好。