Large Language Models (LLMs) have shown promising results in a variety of literary tasks, often using complex memorized details of narration and fictional characters. In this work, we evaluate the ability of Llama-3 at attributing utterances of direct-speech to their speaker in novels. The LLM shows impressive results on a corpus of 28 novels, surpassing published results with ChatGPT and encoder-based baselines by a large margin. We then validate these results by assessing the impact of book memorization and annotation contamination. We found that these types of memorization do not explain the large performance gain, making Llama-3 the new state-of-the-art for quotation attribution in English literature. We release publicly our code and data.

翻译：大型语言模型（LLMs）在多种文学任务中展现出有前景的结果，通常利用对叙事和虚构角色的复杂记忆细节。在本研究中，我们评估了Llama-3在小说中将直接引语归属给说话者的能力。该LLM在包含28部小说的语料库上表现出令人印象深刻的结果，大幅超越了已发表的ChatGPT和基于编码器的基线模型的结果。随后，我们通过评估书籍记忆和标注污染的影响来验证这些结果。我们发现这些类型的记忆无法解释其巨大的性能提升，从而使Llama-3成为英语文学中引语归属任务的新最优方法。我们公开发布了代码和数据。