Today, deep learning is increasingly applied in security-critical situations such as autonomous driving and medical diagnosis. Despite its success, the behavior and robustness of deep networks are not fully understood yet, posing a significant risk. In particular, researchers recently found that neural networks are overly confident in their predictions, even on data they have never seen before. To tackle this issue, one can differentiate two approaches in the literature. One accounts for uncertainty in the predictions, while the second estimates the underlying density of the training data to decide whether a given input is close to the training data, and thus the network is able to perform as expected.In this thesis, we investigate the capabilities of EBMs at the task of fitting the training data distribution to perform detection of out-of-distribution (OOD) inputs. We find that on most datasets, EBMs do not inherently outperform other density estimators at detecting OOD data despite their flexibility. Thus, we additionally investigate the effects of supervision, dimensionality reduction, and architectural modifications on the performance of EBMs. Further, we propose Energy-Prior Network (EPN) which enables estimation of various uncertainties within an EBM for classification, bridging the gap between two approaches for tackling the OOD detection problem. We identify a connection between the concentration parameters of the Dirichlet distribution and the joint energy in an EBM. Additionally, this allows optimization without a held-out OOD dataset, which might not be available or costly to collect in some applications. Finally, we empirically demonstrate that Energy-Prior Network (EPN) is able to detect OOD inputs, datasets shifts, and adversarial examples. Theoretically, EPN offers favorable properties for the asymptotic case when inputs are far from the training data.

翻译：如今，深度学习日益应用于自动驾驶和医学诊断等安全关键场景。尽管取得了成功，但深度网络的行为和鲁棒性尚未完全明确，这构成了重大风险。近年研究发现，神经网络对从未见过的数据也存在过度自信的预测问题。针对该问题，文献中存在两种应对思路：其一通过预测不确定性建模，其二通过估计训练数据的潜在密度分布来判定输入是否接近训练数据，从而确保网络按预期运行。本文研究了能量模型（EBMs）在拟合训练数据分布以进行分布外（OOD）输入检测方面的能力。实验表明，在多数数据集上，尽管EBMs具有灵活性，但其检测OOD数据的性能并未显著优于其他密度估计方法。为此，我们进一步探究了监督信号、降维技术及架构改进对EBM性能的影响。此外，我们提出能量先验网络（EPN），可在分类任务中基于EBM估计多种不确定性，从而弥合OOD检测两种方法之间的鸿沟。我们发现了狄利克雷分布浓度参数与EBM中联合能量之间的关联，该关联允许在不使用留出OOD数据集的情况下进行优化——此类数据集在某些应用中难以获取或收集成本高昂。最后通过实证表明，EPN能够有效检测OOD输入、数据集偏移及对抗样本。从理论层面，EPN在处理远离训练数据的输入时展现出渐进情况下的优越特性。