Recent advances in deep learning have shown that uncertainty estimation is becoming increasingly important in applications such as medical imaging, natural language processing, and autonomous systems. However, accurately quantifying uncertainty remains a challenging problem, especially in regression tasks where the output space is continuous. Deep learning approaches that allow uncertainty estimation for regression problems often converge slowly and yield poorly calibrated uncertainty estimates that can not be effectively used for quantification. Recently proposed post hoc calibration techniques are seldom applicable to regression problems and often add overhead to an already slow model training phase. This work presents a fast calibrated uncertainty estimation method for regression tasks called Likelihood Annealing, that consistently improves the convergence of deep regression models and yields calibrated uncertainty without any post hoc calibration phase. Unlike previous methods for calibrated uncertainty in regression that focus only on low-dimensional regression problems, our method works well on a broad spectrum of regression problems, including high-dimensional regression.Our empirical analysis shows that our approach is generalizable to various network architectures, including multilayer perceptrons, 1D/2D convolutional networks, and graph neural networks, on five vastly diverse tasks, i.e., chaotic particle trajectory denoising, physical property prediction of molecules using 3D atomistic representation, natural image super-resolution, and medical image translation using MRI.

翻译：近期深度学习的进展表明，不确定性估计在医学影像、自然语言处理和自主系统等应用中日益重要。然而，准确量化不确定性仍是一个具有挑战性的问题，尤其是在输出空间连续的回归任务中。允许对回归问题进行不确定性估计的深度学习方法往往收敛缓慢，且生成的不确定性估计校准不佳，难以有效用于量化。近年来提出的后验校准技术很少适用于回归问题，且往往给本已缓慢的模型训练阶段增加额外开销。本研究提出了一种用于回归任务的快速校准不确定性估计方法，称为似然退火（Likelihood Annealing），该方法持续提升深度回归模型的收敛速度，并在无需任何后验校准阶段的情况下生成校准的不确定性。与以往仅关注低维回归问题的校准不确定性方法不同，我们的方法在广泛的回归问题（包括高维回归）中表现良好。我们的实证分析表明，该方法可推广至多种网络架构，包括多层感知机、一维/二维卷积网络和图神经网络，并在五个截然不同的任务上取得成效，即混沌粒子轨迹去噪、利用三维原子表示预测分子物理性质、自然图像超分辨率以及利用MRI进行医学图像翻译。