Large Language Models (LLMs) have demonstrated remarkable performance across various information-seeking and reasoning tasks. These computational systems drive state-of-the-art dialogue systems, such as ChatGPT and Bard. They also carry substantial promise in meeting the growing demands of mental health care, albeit relatively unexplored. As such, this study sought to examine LLMs' capability to generate empathetic responses in conversations that emulate those in a mental health counselling setting. We selected five LLMs: version 3.5 and version 4 of the Generative Pre-training (GPT), Vicuna FastChat-T5, Pathways Language Model (PaLM) version 2, and Falcon-7B-Instruct. Based on a simple instructional prompt, these models responded to utterances derived from the EmpatheticDialogues (ED) dataset. Using three empathy-related metrics, we compared their responses to those from traditional response generation dialogue systems, which were fine-tuned on the ED dataset, along with human-generated responses. Notably, we discovered that responses from the LLMs were remarkably more empathetic in most scenarios. We position our findings in light of catapulting advancements in creating empathetic conversational systems.

翻译：大型语言模型（LLMs）在各类信息查询和推理任务中展现出卓越性能。这些计算系统驱动着ChatGPT和Bard等最先进的对话系统，并在满足日益增长的心理健康护理需求方面具有巨大潜力，尽管这一领域尚待深入探索。为此，本研究旨在考察LLMs在模拟心理健康咨询场景的对话中生成共情回应的能力。我们选取了五种LLMs：生成式预训练模型（GPT）的3.5版本和4版本、Vicuna FastChat-T5、路径语言模型（PaLM）版本2以及Falcon-7B-Instruct。基于一个简单的指令提示，这些模型对来自共情对话数据集（ED）的语句进行了回应。我们采用三项共情相关指标，将其回应与传统的、在ED数据集上微调的回应生成对话系统以及人类生成的回应进行了比较。值得注意的是，我们发现在大多数场景下，LLMs的回应明显更具共情性。我们将研究结果定位在推动共情对话系统创建的前沿进展背景下。