For real-world language applications, detecting an out-of-distribution (OOD) sample is helpful to alert users or reject such unreliable samples. However, modern over-parameterized language models often produce overconfident predictions for both in-distribution (ID) and OOD samples. In particular, language models suffer from OOD samples with a similar semantic representation to ID samples since these OOD samples lie near the ID manifold. A rejection network can be trained with ID and diverse outlier samples to detect test OOD samples, but explicitly collecting auxiliary OOD datasets brings an additional burden for data collection. In this paper, we propose a simple but effective method called Pseudo Outlier Exposure (POE) that constructs a surrogate OOD dataset by sequentially masking tokens related to ID classes. The surrogate OOD sample introduced by POE shows a similar representation to ID data, which is most effective in training a rejection network. Our method does not require any external OOD data and can be easily implemented within off-the-shelf Transformers. A comprehensive comparison with state-of-the-art algorithms demonstrates POE's competitiveness on several text classification benchmarks.

翻译：对于真实世界的语言应用而言，检测分布外（OOD）样本有助于提醒用户或拒绝此类不可靠样本。然而，现代过参数化语言模型通常会对分布内（ID）和OOD样本均产生过度自信的预测。具体而言，语言模型容易受到与ID样本具有相似语义表征的OOD样本影响，因为这些OOD样本位于ID流形附近。虽然可以利用ID样本和多样化的异常样本训练拒绝网络来检测测试阶段的OOD样本，但显式收集辅助性OOD数据集会增加数据收集的额外负担。本文提出一种名为"伪异常样本暴露"（POE）的简单但有效的方法，该方法通过顺序遮蔽与ID类别相关的词元来构建替代性OOD数据集。POE引入的替代性OOD样本具有与ID数据相似的表征，这对训练拒绝网络最为有效。本方法无需任何外部OOD数据，且可轻松集成至现成的Transformer模型中。与最先进算法的全面比较表明，POE在多个文本分类基准测试中具有竞争力。