In recent years Deep Neural Network-based systems are not only increasing in popularity but also receive growing user trust. However, due to the closed-world assumption of such systems, they cannot recognize samples from unknown classes and often induce an incorrect label with high confidence. Presented work looks at the evaluation of methods for Open Set Recognition, focusing on the impact of class imbalance, especially in the dichotomy between known and unknown samples. As an outcome of problem analysis, we present a set of guidelines for evaluation of methods in this field.

翻译：近年来，基于深度神经网络的系统不仅日益普及，也获得了用户越来越多的信任。然而，由于这些系统的封闭世界假设，它们无法识别来自未知类别的样本，且常常以高置信度给出错误标签。本研究聚焦于开放集识别方法的评估，特别关注类别不平衡的影响，尤其是在已知样本与未知样本二分法中的表现。基于问题分析的结果，我们提出了一套该领域方法评估的指南。