Large Language Models (LLMs) are increasingly deployed in human-centric applications, yet they often fail to provide substantive emotional support. While Reinforcement Learning (RL) has been utilized to enhance empathy of LLMs, existing reward models typically evaluate empathy from a single perspective, overlooking the inherently bidirectional interaction nature of empathy between the supporter and seeker as defined by Empathy Cycle theory. To address this limitation, we propose Psychology-grounded Empathetic Reward Modeling (PERM). PERM operationalizes empathy evaluation through a bidirectional decomposition: 1) Supporter perspective, assessing internal resonation and communicative expression; 2) Seeker perspective, evaluating emotional reception. Additionally, it incorporates a bystander perspective to monitor overall interaction quality. Extensive experiments on a widely-used emotional intelligence benchmark and an industrial daily conversation dataset demonstrate that PERM outperforms state-of-the-art baselines by over 10\%. Furthermore, a blinded user study reveals a 70\% preference for our approach, highlighting its efficacy in generating more empathetic responses. Our code, dataset, and models are available at https://github.com/ZhengWwwq/PERM.

翻译：大语言模型（LLMs）越来越多地应用于以人为中心的任务中，但它们往往难以提供实质性的情感支持。尽管强化学习（RL）已被用于增强大语言模型的共情能力，但现有的奖励模型通常仅从单一视角评估共情，忽视了共情周期理论所定义的、支持者与寻求者之间固有的双向互动本质。为克服这一局限，我们提出了基于心理学的共情奖励建模（PERM）。PERM通过双向分解来具体实施共情评估：1）支持者视角，评估内在共鸣与沟通表达；2）寻求者视角，评估情感接收。此外，该方法还引入旁观者视角以监控整体互动质量。在一个广泛使用的情商基准测试和一个工业级日常对话数据集上进行的大量实验表明，PERM的性能优于现有最先进的基线模型超过10%。进一步的盲测用户研究显示，70%的用户更偏好我们的方法，这凸显了其在生成更具共情能力的回复方面的有效性。我们的代码、数据集和模型已在 https://github.com/ZhengWwwq/PERM 公开。