Conversational agents are increasingly used to address emotional needs on top of information needs. One use case of increasing interest are counselling-style mental health and behaviour change interventions, with large language model (LLM)-based approaches becoming more popular. Research in this context so far has been largely system-focused, foregoing the aspect of user behaviour and the impact this can have on LLM-generated texts. To address this issue, we share a dataset containing text-based user interactions related to behaviour change with two GPT-4-based conversational agents collected in a preregistered user study. This dataset includes conversation data, user language analysis, perception measures, and user feedback for LLM-generated turns, and can offer valuable insights to inform the design of such systems based on real interactions.
翻译:摘要:对话代理越来越多地用于在信息需求之外满足情感需求。其中一项日益受到关注的用例是咨询式心理健康与行为改变干预,而基于大语言模型的方法正变得愈发流行。目前该领域的研究主要集中于系统层面,忽视了用户行为因素及其对LLM生成文本的潜在影响。针对这一问题,我们分享了一个数据集,其中包含在一项预先注册的用户研究中,与两个基于GPT-4的对话代理进行文本交互的用户行为改变相关数据。该数据集涵盖对话数据、用户语言分析、感知测量指标以及对LLM生成的轮次反馈信息,可为基于真实交互的系统设计提供有价值的洞见。