Uncertainty quantification (UQ) in deep learning regression is of wide interest, as it supports critical applications including sequential decision making and risk-sensitive tasks. In heteroskedastic regression, where the uncertainty of the target depends on the input, a common approach is to train a neural network that parameterizes the mean and the variance of the predictive distribution. Still, training deep heteroskedastic regression models poses practical challenges in the trade-off between uncertainty quantification and mean prediction, such as optimization difficulties, representation collapse, and variance overfitting. In this work we identify previously undiscussed fallacies and propose a simple and efficient procedure that addresses these challenges jointly by post-hoc fitting a variance model across the intermediate layers of a pretrained network on a hold-out dataset. We demonstrate that our method achieves on-par or state-of-the-art uncertainty quantification on several molecular graph datasets, without compromising mean prediction accuracy and remaining cheap to use at prediction time.

翻译：深度学习回归中的不确定性量化具有广泛意义，因其支持包括序列决策与风险敏感任务在内的关键应用。在异方差回归中，目标的不确定性取决于输入，常见方法是训练参数化预测分布均值与方差的神经网络。然而，深度异方差回归模型的训练在不确定性量化与均值预测的权衡中仍面临实际挑战，例如优化困难、表示坍缩及方差过拟合。本工作识别了先前未被讨论的认知误区，并提出一种简单高效的解决方案：通过在预训练网络的中间层上使用保留数据集对方差模型进行事后拟合，从而协同应对这些挑战。我们在多个分子图数据集上证明，该方法在保持预测时计算成本低廉且不损害均值预测精度的前提下，实现了可比或最先进的不确定性量化性能。