We introduce a Reinforcement Learning Psychotherapy AI Companion that generates topic recommendations for therapists based on patient responses. The system uses Deep Reinforcement Learning (DRL) to generate multi-objective policies for four different psychiatric conditions: anxiety, depression, schizophrenia, and suicidal cases. We present our experimental results on the accuracy of recommended topics using three different scales of working alliance ratings: task, bond, and goal. We show that the system is able to capture the real data (historical topics discussed by the therapists) relatively well, and that the best performing models vary by disorder and rating scale. To gain interpretable insights into the learned policies, we visualize policy trajectories in a 2D principal component analysis space and transition matrices. These visualizations reveal distinct patterns in the policies trained with different reward signals and trained on different clinical diagnoses. Our system's success in generating DIsorder-Specific Multi-Objective Policies (DISMOP) and interpretable policy dynamics demonstrates the potential of DRL in providing personalized and efficient therapeutic recommendations.

翻译：我们提出了一种基于强化学习的心理治疗AI伴侣，可根据患者反馈为治疗师生成主题推荐。该系统采用深度强化学习（DRL）为四种精神疾病（焦虑症、抑郁症、精神分裂症和自杀倾向病例）生成多目标策略。我们展示了通过三种不同工作联盟量表（任务、纽带和目标）对推荐主题准确性的实验结果。研究表明，该系统能较好地捕捉真实数据（治疗师实际讨论的历史主题），且最优模型因疾病类型和评分量表而异。为获得对所学策略的可解释性洞察，我们在二维主成分分析空间和转移矩阵中可视化策略轨迹。这些可视化揭示了基于不同奖励信号训练且面向不同临床诊断的策略呈现的独特模式。本系统成功生成的疾病特异性多目标策略（DISMOP）及可解释策略动力学，证明了深度强化学习在提供个性化高效治疗建议方面的潜力。