Recent years have seen increasing concerns about the unsafe response generation of large-scale dialogue systems, where agents will learn offensive or biased behaviors from the real-world corpus. Some methods are proposed to address the above issue by detecting and replacing unsafe training examples in a pipeline style. Though effective, they suffer from a high annotation cost and adapt poorly to unseen scenarios as well as adversarial attacks. Besides, the neglect of providing safe responses (e.g. simply replacing with templates) will cause the information-missing problem of dialogues. To address these issues, we propose an unsupervised pseudo-label sampling method, TEMP, that can automatically assign potential safe responses. Specifically, our TEMP method groups responses into several clusters and samples multiple labels with an adaptively sharpened sampling strategy, inspired by the observation that unsafe samples in the clusters are usually few and distribute in the tail. Extensive experiments in chitchat and task-oriented dialogues show that our TEMP outperforms state-of-the-art models with weak supervision signals and obtains comparable results under unsupervised learning settings.

翻译：近年来，大规模对话系统生成的不安全回复问题日益引发关注，即智能体可能从真实世界语料库中学习到具有攻击性或偏见的行为。现有方法通常采用流水线式策略，通过检测并替换不安全训练样本来解决上述问题。尽管这些方法有效，但存在标注成本高、对未见场景和对抗攻击的泛化能力差等缺陷。此外，忽略提供安全回复（如仅用模板简单替换）会导致对话信息缺失问题。为解决上述挑战，我们提出一种无监督伪标签采样方法TEMP，可自动分配潜在的安全回复。具体而言，受聚类中不安全样本通常数量较少且分布在尾部的现象启发，我们的TEMP方法将回复归入若干聚类，并采用自适应锐化采样策略获取多个标签。在闲聊和任务型对话上的大量实验表明，TEMP方法在弱监督信号下优于现有最优模型，并在无监督学习设置下获得可比的性能。