Detecting out-of-scope user utterances is essential for task-oriented dialogues and intent classification. Current methodologies face difficulties with the unpredictable distribution of outliers and often rely on assumptions about data distributions. We present the Dual Encoder for Threshold-Based Re-Classification (DETER) to address these challenges. This end-to-end framework efficiently detects out-of-scope intents without requiring assumptions on data distributions or additional post-processing steps. The core of DETER utilizes dual text encoders, the Universal Sentence Encoder (USE) and the Transformer-based Denoising AutoEncoder (TSDAE), to generate user utterance embeddings, which are classified through a branched neural architecture. Further, DETER generates synthetic outliers using self-supervision and incorporates out-of-scope phrases from open-domain datasets. This approach ensures a comprehensive training set for out-of-scope detection. Additionally, a threshold-based re-classification mechanism refines the model's initial predictions. Evaluations on the CLINC-150, Stackoverflow, and Banking77 datasets demonstrate DETER's efficacy. Our model outperforms previous benchmarks, increasing up to 13% and 5% in F1 score for known and unknown intents on CLINC-150 and Stackoverflow, and 16% for known and 24% % for unknown intents on Banking77. The source code has been released at https://github.com/Hossam-Mohammed-tech/Intent\_Classification\_OOS.
翻译:在任务导向对话与意图分类中,检测用户话语是否超出预设范围至关重要。现有方法因异常值分布难以预测而面临挑战,且常依赖于对数据分布的假设。本文提出基于阈值重分类的双重编码器(DETER)以应对这些问题。该端到端框架无需数据分布假设或额外后处理步骤,即可有效检测域外意图。DETER的核心采用双重文本编码器——通用语句编码器(USE)与基于Transformer的去噪自编码器(TSDAE)——生成用户话语嵌入表示,并通过分支神经架构进行分类。此外,DETER通过自监督生成合成异常样本,并整合开放域数据集中的域外短语,从而构建了用于域外检测的全面训练集。基于阈值的重分类机制进一步优化了模型的初始预测结果。在CLINC-150、Stackoverflow和Banking77数据集上的评估验证了DETER的有效性:在CLINC-150与Stackoverflow数据集上,模型对已知意图和未知意图的F1分数分别提升最高达13%与5%;在Banking77数据集上,对已知意图和未知意图的F1分数分别提升16%与24%。源代码已发布于https://github.com/Hossam-Mohammed-tech/Intent\_Classification\_OOS。