Bayesian regression determines model parameters by minimizing the expected loss, an upper bound to the true generalization error. However, the loss ignores misspecification, where models are imperfect. Parameter uncertainties from Bayesian regression are thus significantly underestimated and vanish in the large data limit. This is particularly problematic when building models of low- noise, or near-deterministic, calculations, as the main source of uncertainty is neglected. We analyze the generalization error of misspecified, near-deterministic surrogate models, a regime of broad relevance in science and engineering. We show posterior distributions must cover every training point to avoid a divergent generalization error and design an ansatz that respects this constraint, which for linear models incurs minimal overhead. This is demonstrated on model problems before application to thousand dimensional datasets in atomistic machine learning. Our efficient misspecification-aware scheme gives accurate prediction and bounding of test errors where existing schemes fail, allowing this important source of uncertainty to be incorporated in computational workflows.

翻译：贝叶斯回归通过最小化期望损失来确定模型参数，该损失是真泛化误差的上界。然而，这种损失忽略了模型不完美时的设定偏差。贝叶斯回归得到的参数不确定性因此被显著低估，并在大数据极限下消失。这一问题在构建低噪声或近确定性计算模型时尤为突出——因为不确定性主要来源被忽视。我们分析了科学和工程领域广泛相关的近确定性不完美代理模型的泛化误差，证明后验分布必须覆盖每个训练点以避免泛化误差发散，并设计了一个满足该约束的拟设方法（对线性模型而言仅需极小额外开销）。通过模型问题验证后，该方法被应用于原子机器学习中数千维数据集。我们提出的高效设定偏差感知方案能在现有方案失效时准确预测和界定测试误差，从而将这一重要不确定性来源纳入计算工作流。