Providing dialogue agents with a profile representation can improve their consistency and coherence, leading to better conversations. However, current profile-based dialogue datasets for training such agents contain either explicit profile representations that are simple and dialogue-specific, or implicit representations that are difficult to collect. In this work, we propose a unified framework in which we bring together both standard and more sophisticated profile representations by creating a new resource where each dialogue is aligned with all possible speaker representations such as communication style, biographies, and personality. This framework allows to test several baselines built using generative language models with several profile configurations. The automatic evaluation shows that profile-based models have better generalisation capabilities than models trained on dialogues only, both in-domain and cross-domain settings. These results are consistent for fine-tuned models and instruction-based LLMs. Additionally, human evaluation demonstrates a clear preference for generations consistent with both profile and context. Finally, to account for possible privacy concerns, all experiments are done under two configurations: inter-character and intra-character. In the former, the LM stores the information about the character in its internal representation, while in the latter, the LM does not retain any personal information but uses it only at inference time.
翻译:为对话代理提供角色特征表示可提升其一致性与连贯性,从而改善对话质量。然而,当前用于训练此类代理的基于角色的对话数据集中,显式角色表示虽简单但局限于特定对话场景,而隐式表示则难以采集。本研究提出统一框架,通过创建新资源使每个对话与所有可能的说话人表示(如沟通风格、传记、性格)对齐,整合了标准角色表示与更复杂的角色表示。该框架允许在多种角色配置下,测试基于生成语言模型构建的多个基线模型。自动评估表明,无论在领域内还是跨领域场景中,基于角色的模型相较于仅基于对话训练的模型具有更强的泛化能力,该结论在微调模型与基于指令的大语言模型中均一致。此外,人工评估显示,人们更倾向于生成内容同时匹配角色与上下文语境的结果。最后,为应对隐私问题,所有实验均在"角色间"与"角色内"两种配置下进行:前者中语言模型将角色信息存储于内部表征,后者中语言模型不保留任何个人信息,仅在推理时使用该信息。