The development of chatbots requires collecting a large number of human-chatbot dialogues to reflect the breadth of users' sociodemographic backgrounds and conversational goals. However, the resource requirements to conduct the respective user studies can be prohibitively high and often only allow for a narrow analysis of specific dialogue goals and participant demographics. In this paper, we propose LLM-Roleplay: a goal-oriented, persona-based method to automatically generate diverse multi-turn dialogues simulating human-chatbot interaction. LLM-Roleplay can be applied to generate dialogues with any type of chatbot and uses large language models (LLMs) to play the role of textually described personas. To validate our method we collect natural human-chatbot dialogues from different sociodemographic groups and conduct a human evaluation to compare real human-chatbot dialogues with our generated dialogues. We compare the abilities of state-of-the-art LLMs in embodying personas and holding a conversation and find that our method can simulate human-chatbot dialogues with a high indistinguishability rate.

翻译：聊天机器人的开发需要收集大量人机对话数据，以反映用户社会人口背景与对话目标的多样性。然而，开展相应用户研究所需的资源成本往往过高，且通常只能针对特定对话目标与参与者群体进行有限分析。本文提出LLM-Roleplay：一种基于目标导向与人物设定的自动化方法，用于生成模拟人机交互的多样化多轮对话。该方法可适配任意类型的聊天机器人，并利用大语言模型（LLM）扮演文本描述的人物角色。为验证方法的有效性，我们收集了来自不同社会人口群体的真实人机对话数据，并通过人工评估将真实对话与生成对话进行对比。通过比较前沿LLM在角色拟真与对话维持方面的能力，我们发现该方法能以较高的不可区分率有效模拟人机对话。