Accurate uncertainty estimation is vital to trustworthy machine learning, yet uncertainties typically have to be learned for each task anew. This work introduces the first pretrained uncertainty modules for vision models. Similar to standard pretraining this enables the zero-shot transfer of uncertainties learned on a large pretraining dataset to specialized downstream datasets. We enable our large-scale pretraining on ImageNet-21k by solving a gradient conflict in previous uncertainty modules and accelerating the training by up to 180x. We find that the pretrained uncertainties generalize to unseen datasets. In scrutinizing the learned uncertainties, we find that they capture aleatoric uncertainty, disentangled from epistemic components. We demonstrate that this enables safe retrieval and uncertainty-aware dataset visualization. To encourage applications to further problems and domains, we release all pretrained checkpoints and code under https://github.com/mkirchhof/url .

翻译：准确的不确定性估计对于可信赖的机器学习至关重要，然而不确定性通常需要针对每个任务从头学习。本文首次提出了针对视觉模型的预训练不确定性模块。类似于标准预训练，这使得在大型预训练数据集上学到的不确定性能够零样本迁移到专门的下游数据集。通过解决先前不确定性模块中的梯度冲突并将训练速度提升180倍，我们能够在ImageNet-21k上实现大规模预训练。我们发现预训练的不确定性能够泛化到未见数据集。通过仔细审视学习到的不确定性，发现其捕获了与认知成分解耦的偶然不确定性。我们证明这能够实现安全检索和不确定性感知的数据集可视化。为鼓励在更多问题和领域的应用，我们在https://github.com/mkirchhof/url 上开放所有预训练检查点和代码。