NSFW (Not Safe for Work) content, in the context of a dialogue, can have severe side effects on users in open-domain dialogue systems. However, research on detecting NSFW language, especially sexually explicit content, within a dialogue context has significantly lagged behind. To address this issue, we introduce CensorChat, a dialogue monitoring dataset aimed at NSFW dialogue detection. Leveraging knowledge distillation techniques involving GPT-4 and ChatGPT, this dataset offers a cost-effective means of constructing NSFW content detectors. The process entails collecting real-life human-machine interaction data and breaking it down into single utterances and single-turn dialogues, with the chatbot delivering the final utterance. ChatGPT is employed to annotate unlabeled data, serving as a training set. Rationale validation and test sets are constructed using ChatGPT and GPT-4 as annotators, with a self-criticism strategy for resolving discrepancies in labeling. A BERT model is fine-tuned as a text classifier on pseudo-labeled data, and its performance is assessed. The study emphasizes the importance of AI systems prioritizing user safety and well-being in digital conversations while respecting freedom of expression. The proposed approach not only advances NSFW content detection but also aligns with evolving user protection needs in AI-driven dialogues.

翻译：NSFW（不宜工作场合）内容在开放域对话系统中可能对用户产生严重的负面影响。然而，在对话语境中检测NSFW语言（尤其是露骨色情内容）的研究明显滞后。为解决这一问题，我们提出了CensorChat——一个面向NSFW对话检测的对话监控数据集。该数据集利用涉及GPT-4和ChatGPT的知识蒸馏技术，提供了一种经济高效构建NSFW内容检测器的方法。具体流程包括收集真实人机交互数据，并将其分解为单轮次话语和单轮对话（其中聊天机器人输出最终话语），采用ChatGPT对未标注数据进行标注以构建训练集。通过ChatGPT和GPT-4作为标注器，结合自我批评策略解决标注分歧，构建了理由验证集和测试集。基于伪标注数据微调BERT模型作为文本分类器，并评估其性能。本研究强调AI系统在数字对话中需在尊重言论自由的同时优先保障用户安全与福祉。所提方法不仅推进了NSFW内容检测技术，也契合了AI驱动对话中不断演进的用户保护需求。