An emotional support conversation system aims to alleviate users' emotional distress and assist them in addressing their challenges. To generate supportive responses, it is critical to consider multiple factors such as empathy, support strategies, and response coherence, as established in prior methods. Nonetheless, previous models occasionally generate unhelpful responses, which intend to provide support but display counterproductive effects. According to psychology and communication theories, poor performance in just one contributing factor might cause a response to be unhelpful. From the model training perspective, since these models have not been exposed to unhelpful responses during their training phase, they are unable to distinguish if the tokens they generate might result in unhelpful responses during inference. To address this issue, we introduce a novel model-agnostic framework named mitigating unhelpfulness with multifaceted AI feedback for emotional support (Muffin). Specifically, Muffin employs a multifaceted AI feedback module to assess the helpfulness of responses generated by a specific model with consideration of multiple factors. Using contrastive learning, it then reduces the likelihood of the model generating unhelpful responses compared to the helpful ones. Experimental results demonstrate that Muffin effectively mitigates the generation of unhelpful responses while slightly increasing response fluency and relevance.

翻译：情感支持对话系统旨在缓解用户的情感困扰并协助他们应对挑战。为了生成支持性回应，如先前方法所确立，考虑共情、支持策略和回应连贯性等多重因素至关重要。然而，先前的模型偶尔会生成无帮助的回应，这些回应本意提供支持却产生反效果。根据心理学和传播理论，仅在某一个影响因素上表现不佳就可能导致回应无帮助。从模型训练的角度来看，由于这些模型在训练阶段未接触过无帮助回应，它们在推理过程中无法区分所生成的标记是否可能导致无帮助回应。为解决此问题，我们引入了一种新颖的模型无关框架，名为通过多面AI反馈减轻情感支持中的无帮助性（Muffin）。具体而言，Muffin采用多面AI反馈模块，考虑多重因素评估特定模型所生成回应的帮助性。通过对比学习，它随后降低模型生成无帮助回应的可能性，相较于有帮助的回应。实验结果表明，Muffin有效减轻了无帮助回应的生成，同时轻微提升了回应的流畅性和相关性。