Uncertainty estimation in machine learning has traditionally focused on the prediction stage, aiming to quantify confidence in model outputs while treating learned representations as deterministic and reliable by default. In this work, we challenge this implicit assumption and argue that reliability should be regarded as a first-class property of learned representations themselves. We propose a principled framework for reliable representation learning that explicitly models representation-level uncertainty and leverages structural constraints as inductive biases to regularize the space of feasible representations. Our approach introduces uncertainty-aware regularization directly in the representation space, encouraging representations that are not only predictive but also stable, well-calibrated, and robust to noise and structural perturbations. Structural constraints, such as sparsity, relational structure, or feature-group dependencies, are incorporated to define meaningful geometry and reduce spurious variability in learned representations, without assuming fully correct or noise-free structure. Importantly, the proposed framework is independent of specific model architectures and can be integrated with a wide range of representation learning methods.

翻译：机器学习中的不确定性估计传统上聚焦于预测阶段，旨在量化模型输出的置信度，同时默认将学习到的表征视为确定且可靠的。本研究挑战了这一隐含假设，主张可靠性应被视为学习表征本身的一阶属性。我们提出一个用于可靠表征学习的原理性框架，该框架显式建模表征层面的不确定性，并利用结构约束作为归纳偏置来正则化可行表征空间。我们的方法直接在表征空间中引入不确定性感知正则化，鼓励获得的表征不仅具有预测性，同时具备稳定性、良好校准性以及对噪声与结构扰动的鲁棒性。通过引入稀疏性、关系结构或特征组依赖等结构约束，可在不假定结构完全正确或无噪声的前提下，定义有意义的几何结构并减少学习表征中的伪变异性。重要的是，所提框架独立于特定模型架构，可与广泛的表征学习方法相结合。