Sampling-based methods, e.g., Deep Ensembles and Bayesian Neural Nets have become promising approaches to improve the quality of uncertainty estimation and robust generalization. However, they suffer from a large model size and high latency at test-time, which limits the scalability needed for low-resource devices and real-time applications. To resolve these computational issues, we propose Density-Softmax, a sampling-free deterministic framework via combining a density function built on a Lipschitz-constrained feature extractor with the softmax layer. Theoretically, we show that our model is the solution of minimax uncertainty risk and is distance-aware on feature space, thus reducing the over-confidence of the standard softmax under distribution shifts. Empirically, our method enjoys competitive results with state-of-the-art techniques in terms of uncertainty and robustness, while having a lower number of model parameters and a lower latency at test-time.

翻译：基于采样的方法（如深度集成与贝叶斯神经网络）已成为提升不确定性估计质量与鲁棒泛化能力的重要途径。然而，这些方法存在模型体积大、测试时延迟高的问题，限制了其在低资源设备与实时应用中的可扩展性。为解决这些计算瓶颈，本文提出密度Softmax——一种免采样的确定性框架，通过将基于Lipschitz约束特征提取器构建的密度函数与softmax层相结合实现。理论分析表明，该模型是最小化最大不确定性风险的解，且在特征空间上具有距离感知能力，从而缓解了标准softmax在分布偏移下的过度自信问题。实验证明，本方法在不确定性与鲁棒性指标上与最先进技术具有可比性，同时显著减少了模型参数量并降低了测试时延迟。