Bayesian inference is often utilized for uncertainty quantification tasks. A recent analysis by Xu and Raginsky 2022 rigorously decomposed the predictive uncertainty in Bayesian inference into two uncertainties, called aleatoric and epistemic uncertainties, which represent the inherent randomness in the data-generating process and the variability due to insufficient data, respectively. They analyzed those uncertainties in an information-theoretic way, assuming that the model is well-specified and treating the model's parameters as latent variables. However, the existing information-theoretic analysis of uncertainty cannot explain the widely believed property of uncertainty, known as the sensitivity between the test and training data. It implies that when test data are similar to training data in some sense, the epistemic uncertainty should become small. In this work, we study such uncertainty sensitivity using our novel decomposition method for the predictive uncertainty. Our analysis successfully defines such sensitivity using information-theoretic quantities. Furthermore, we extend the existing analysis of Bayesian meta-learning and show the novel sensitivities among tasks for the first time.

翻译：贝叶斯推断常被用于不确定性量化任务。Xu与Raginsky（2022）的最新分析将贝叶斯推断中的预测不确定性严格分解为两类不确定性——偶然不确定性与认知不确定性，分别代表数据生成过程中的固有随机性以及因数据不足导致的变异性。他们采用信息论方法分析这些不确定性，假设模型设定正确并将模型参数视为隐变量。然而，现有不确定性信息论分析无法解释被广泛认可的不确定性性质——即测试数据与训练数据之间的敏感性。该性质表明，当测试数据在某种意义上与训练数据相似时，认知不确定性应趋于减小。本研究通过提出的预测不确定性分解方法，对这种不确定性敏感性进行深入探究。我们的分析成功利用信息论量定义了此类敏感性。此外，我们将贝叶斯元学习的现有分析进行拓展，首次揭示了任务间的新颖敏感性特征。