An emotional support conversation system aims to alleviate users' emotional distress and assist them in addressing their challenges. To generate supportive responses, it is critical to consider multiple factors such as empathy, support strategies, and response coherence, as established in prior methods. Nonetheless, previous models occasionally generate unhelpful responses, which intend to provide support but display counterproductive effects. According to psychology and communication theories, poor performance in just one contributing factor might cause a response to be unhelpful. From the model training perspective, since these models have not been exposed to unhelpful responses during their training phase, they are unable to distinguish if the tokens they generate might result in unhelpful responses during inference. To address this issue, we introduce a novel model-agnostic framework named mitigating unhelpfulness with multifaceted AI feedback for emotional support (Muffin). Specifically, Muffin employs a multifaceted AI feedback module to assess the helpfulness of responses generated by a specific model with consideration of multiple factors. Using contrastive learning, it then reduces the likelihood of the model generating unhelpful responses compared to the helpful ones. Experimental results demonstrate that Muffin effectively mitigates the generation of unhelpful responses while slightly increasing response fluency and relevance.

翻译：情感支持对话系统旨在缓解用户的情感困扰，并协助其应对挑战。为了生成支持性回答，需要综合考虑多个因素，如共情、支持策略及回答连贯性——正如先前方法所确立的那样。然而，以往模型偶尔会生成无益回答，这些回答本意是提供支持，却产生了适得其反的效果。根据心理学与传播学理论，单一影响因素的表现不佳可能导致回答失去效用。从模型训练的角度看，由于这些模型在训练阶段未曾接触过无益回答，它们无法在推理过程中区分自己生成的词元是否会导致无益回答。为解决这一问题，我们提出了一种新颖的模型无关框架，名为基于多面AI反馈缓解情感支持中的无益性（Muffin）。具体而言，Muffin采用多面AI反馈模块，在考虑多个因素的前提下评估特定模型生成回答的有益性。随后，通过对比学习，它降低模型生成无益回答相对于有益回答的概率。实验结果表明，Muffin有效缓解了无益回答的生成，同时略微提升了回答的流畅度与相关性。