Successful detection of Out-of-Distribution (OoD) data is becoming increasingly important to ensure safe deployment of neural networks. One of the main challenges in OoD detection is that neural networks output overconfident predictions on OoD data, make it difficult to determine OoD-ness of data solely based on their predictions. Outlier exposure addresses this issue by introducing an additional loss that encourages low-confidence predictions on OoD data during training. While outlier exposure has shown promising potential in improving OoD detection performance, all previous studies on outlier exposure have been limited to utilizing visual outliers. Drawing inspiration from the recent advancements in vision-language pre-training, this paper venture out to the uncharted territory of textual outlier exposure. First, we uncover the benefits of using textual outliers by replacing real or virtual outliers in the image-domain with textual equivalents. Then, we propose various ways of generating preferable textual outliers. Our extensive experiments demonstrate that generated textual outliers achieve competitive performance on large-scale OoD and hard OoD benchmarks. Furthermore, we conduct empirical analyses of textual outliers to provide primary criteria for designing advantageous textual outliers: near-distribution, descriptiveness, and inclusion of visual semantics.

翻译：分布外（OoD）数据的成功检测对于确保神经网络的安全部署日益重要。OoD检测的主要挑战之一是神经网络对OoD数据会输出过度自信的预测，使得仅基于预测结果难以判断数据的OoD属性。异常值暴露通过引入额外损失函数来解决此问题，该损失函数在训练过程中鼓励对OoD数据做出低置信度预测。尽管异常值暴露在提升OoD检测性能方面展现出巨大潜力，但此前所有相关研究均局限于使用视觉异常值。受近期视觉-语言预训练进展的启发，本文首次探索文本异常值暴露这一未知领域。首先，我们发现通过将图像域中的真实或虚拟异常值替换为等效文本异常值可带来性能优势。随后，我们提出多种生成优选文本异常值的方法。大量实验表明，生成的文本异常值在大型OoD基准测试和困难OoD基准测试中均能达到具有竞争力的性能。此外，我们对文本异常值进行实证分析，为设计有利的文本异常值提供主要准则：近分布性、描述性和视觉语义包含性。