Detecting out-of-distribution (OOD) data is critical to building reliable machine learning systems in the open world. Among the existing OOD detection methods, ReAct is famous for its simplicity and efficiency, and has good theoretical analysis. The gap between ID data and OOD data is enlarged by clipping the larger activation value. But the question is, is this operation optimal? Is there a better way to expand the spacing between ID samples and OOD samples in theory? Driven by these questions, we view the optimal activation function modification from the perspective of functional extremum and propose the Variational Recified Acitvations (VRA) method. In order to make our method easy to practice, we further propose several VRA variants. To verify the effectiveness of our method, we conduct experiments on many benchmark datasets. Experimental results demonstrate that our method outperforms existing state-of-the-art approaches. Meanwhile, our method is easy to implement and does not require additional OOD data or fine-tuning process. We can realize OOD detection in only one forward pass.

翻译：检测分布外数据对于在开放世界中构建可靠的机器学习系统至关重要。在现有分布外检测方法中，ReAct以其简洁高效而著称，并具有良好的理论分析基础。通过截断较大激活值来扩大分布内与分布外数据之间的差异。但问题在于，此操作是否最优？是否存在理论层面更优的方法来扩大分布内样本与分布外样本的间隔？受这些问题驱动，我们从函数极值角度审视最优激活函数修改问题，并提出变分整流激活方法。为使方法易于实践，我们进一步提出了若干VRA变体。为验证方法有效性，我们在多个基准数据集上开展实验。实验结果表明，我们的方法优于现有最先进方法。同时，该方法实现简便，无需额外分布外数据或微调过程，仅需一次前向传播即可完成分布外检测。