Virtual Mental Health Assistants (VMHAs) have become a prevalent method for receiving mental health counseling in the digital healthcare space. An assistive counseling conversation commences with natural open-ended topics to familiarize the client with the environment and later converges into more fine-grained domain-specific topics. Unlike other conversational systems, which are categorized as open-domain or task-oriented systems, VMHAs possess a hybrid conversational flow. These counseling bots need to comprehend various aspects of the conversation, such as dialogue-acts, intents, etc., to engage the client in an effective conversation. Although the surge in digital health research highlights applications of many general-purpose response generation systems, they are barely suitable in the mental health domain -- the prime reason is the lack of understanding in mental health counseling. Moreover, in general, dialogue-act guided response generators are either limited to a template-based paradigm or lack appropriate semantics. To this end, we propose READER -- a REsponse-Act guided reinforced Dialogue genERation model for the mental health counseling conversations. READER is built on transformer to jointly predict a potential dialogue-act d(t+1) for the next utterance (aka response-act) and to generate an appropriate response u(t+1). Through the transformer-reinforcement-learning (TRL) with Proximal Policy Optimization (PPO), we guide the response generator to abide by d(t+1) and ensure the semantic richness of the responses via BERTScore in our reward computation. We evaluate READER on HOPE, a benchmark counseling conversation dataset and observe that it outperforms several baselines across several evaluation metrics -- METEOR, ROUGE, and BERTScore. We also furnish extensive qualitative and quantitative analyses on results, including error analysis, human evaluation, etc.

翻译：虚拟心理健康助手已成为数字医疗领域接受心理健康咨询的主流方式。辅助性咨询对话从开放式自然话题开始，帮助来访者熟悉环境，随后逐步聚焦到更细粒度的特定领域话题。与其他被划分为开放域或任务导向型系统的对话系统不同，虚拟心理健康助手具有混合对话流特征。这类咨询机器人需要理解对话的多个维度（如对话行为、意图等），才能与来访者进行有效互动。尽管数字健康研究的蓬勃发展推动了通用型回复生成系统的广泛应用，但这些系统在心理健康领域并不适用——主要原因在于缺乏对心理健康咨询的理解。此外，普遍存在的对话行为引导型回复生成系统要么局限于模板范式，要么缺乏恰当的语义表达。为此，我们提出READER——面向心理健康咨询对话的响应行为引导增强对话生成模型。READER基于Transformer架构，可联合预测下一轮对话的潜在对话行为d(t+1)（即响应行为）并生成恰当回复u(t+1)。通过采用近端策略优化的Transformer强化学习技术，我们引导回复生成器遵循d(t+1)，并在奖励计算中借助BERTScore确保回复语义丰富性。我们在基准咨询对话数据集HOPE上评估READER，观察到其在多个评估指标（METEOR、ROUGE和BERTScore）上均优于多个基线模型。我们还提供了详尽的定性与定量分析结果，包括错误分析、人工评估等。