Epistemic uncertainty is often viewed as a reducible uncertainty that vanishes with increasing data. This perspective implicitly assumes parameter identifiability and equates epistemic uncertainty with predictive variability. In overparametrized neural networks, however, model parameters are typically non-identifiable due to symmetries and redundant representations. As a consequence, substantial parameter uncertainty can persist even when the underlying function is fully identified. In this work, we analyze epistemic uncertainty through the lens of non-identifiability and characterize both discrete and continuous sources of residual uncertainty. Focusing on one-hidden-layer ReLU networks, we thoroughly analyze the resulting posterior structure and validate our theoretical insights through empirical studies.

翻译：认知不确定性常被视为一种可约减的不确定性，随数据增加而消失。这一观点隐含地假设了参数可辨识性，并将认知不确定性等同于预测变异性。然而，在过参数化神经网络中，由于对称性和冗余表示，模型参数通常不可辨识。因此，即使潜在函数被完全识别，显著的参数不确定性仍可能持续存在。本文从不可辨识性的角度分析认知不确定性，并刻画了离散与连续两类残差不确定性。我们以单隐层ReLU网络为焦点，深入分析了其产生的后验结构，并通过实证研究验证了理论洞见。