Considerable efforts have been invested in augmenting the role-playing proficiency of open-source large language models (LLMs) by emulating proprietary counterparts. Nevertheless, we posit that LLMs inherently harbor role-play capabilities, owing to the extensive knowledge of characters and potential dialogues ingrained in their vast training corpora. Thus, in this study, we introduce Ditto, a self-alignment method for role-play. Ditto capitalizes on character knowledge, encouraging an instruction-following LLM to simulate role-play dialogues as a variant of reading comprehension. This method creates a role-play training set comprising 4,000 characters, surpassing the scale of currently available datasets by tenfold regarding the number of roles. Subsequently, we fine-tune the LLM using this self-generated dataset to augment its role-playing capabilities. Upon evaluating our meticulously constructed and reproducible role-play benchmark and the roleplay subset of MT-Bench, Ditto, in various parameter scales, consistently maintains a consistent role identity and provides accurate role-specific knowledge in multi-turn role-play conversations. Notably, it outperforms all open-source role-play baselines, showcasing performance levels comparable to advanced proprietary chatbots. Furthermore, we present the first comprehensive cross-supervision alignment experiment in the role-play domain, revealing that the intrinsic capabilities of LLMs confine the knowledge within role-play. Meanwhile, the role-play styles can be easily acquired with the guidance of smaller models. We open-source related resources at https://github.com/OFA-Sys/Ditto.

翻译：大量研究致力于通过模仿专有模型来增强开源大型语言模型（LLMs）的角色扮演能力。然而，我们认为LLMs本身便天生具备角色扮演能力，这是因为其庞大的训练语料中蕴含了广泛的人物知识与潜在对话模式。因此，本研究提出Ditto——一种用于角色扮演的自对齐方法。Ditto利用人物知识，鼓励遵循指令的LLM将角色扮演对话模拟为阅读理解的一种变体。该方法构建了一个包含4,000个人物的角色扮演训练集，其角色数量规模较现有数据集提升十倍。随后，我们使用该自生成数据集微调LLM以增强其角色扮演能力。在精心构建且可复现的角色扮演基准测试以及MT-Bench的角色扮演子集评估中，不同参数规模的Ditto在多轮角色扮演对话中均能保持稳定的角色身份，并提供准确的角色特定知识。值得注意的是，它超越了所有开源角色扮演基线模型，展现出与先进专有聊天机器人相当的性能水平。此外，我们首次在角色扮演领域开展了全面的跨监督对齐实验，揭示出LLMs的内在能力限制了角色扮演中的知识范围，而角色扮演风格则可在较小模型的引导下轻松习得。相关资源已在https://github.com/OFA-Sys/Ditto开源。