Implementing effective control mechanisms to ensure the proper functioning and security of deployed NLP models, from translation to chatbots, is essential. A key ingredient to ensure safe system behaviour is Out-Of-Distribution (OOD) detection, which aims to detect whether an input sample is statistically far from the training distribution. Although OOD detection is a widely covered topic in classification tasks, most methods rely on hidden features output by the encoder. In this work, we focus on leveraging soft-probabilities in a black-box framework, i.e. we can access the soft-predictions but not the internal states of the model. Our contributions include: (i) RAINPROOF a Relative informAItioN Projection OOD detection framework; and (ii) a more operational evaluation setting for OOD detection. Surprisingly, we find that OOD detection is not necessarily aligned with task-specific measures. The OOD detector may filter out samples well processed by the model and keep samples that are not, leading to weaker performance. Our results show that RAINPROOF provides OOD detection methods more aligned with task-specific performance metrics than traditional OOD detectors.

翻译：实施有效控制机制以确保部署的NLP模型（从翻译到聊天机器人）的正常运行与安全性至关重要。确保系统安全行为的关键要素是过分布（OOD）检测，其目标在于判断输入样本是否在统计上显著偏离训练分布。尽管OOD检测在分类任务中已被广泛讨论，但大多数方法依赖于编码器输出的隐藏特征。本研究聚焦于在黑盒框架下利用软概率，即我们能够访问模型输出的软预测结果，但无法获取其内部状态。我们的贡献包括：（i）RAINPROOF——一种基于相对信息投影的OOD检测框架；（ii）一种更具操作性的OOD检测评估方案。令人惊讶的是，我们发现OOD检测与特定任务指标之间未必存在关联性。OOD检测器可能过滤掉模型处理良好的样本，同时保留处理效果不佳的样本，从而导致性能下降。实验结果表明，与传统OOD检测方法相比，RAINPROOF提供的OOD检测方法与任务特定性能指标具有更高的一致性。