Quantifying predictive uncertainty is essential for real world machine learning applications, especially in scenarios requiring reliable and interpretable predictions. Many common parametric approaches rely on neural networks to estimate distribution parameters by optimizing the negative log likelihood. However, these methods often encounter challenges like training instability and mode collapse, leading to poor estimates of the mean and variance of the target output distribution. In this work, we propose the Neural Energy Gaussian Mixture Model (NE-GMM), a novel framework that integrates Gaussian Mixture Model (GMM) with Energy Score (ES) to enhance predictive uncertainty quantification. NE-GMM leverages the flexibility of GMM to capture complex multimodal distributions and leverages the robustness of ES to ensure well calibrated predictions in diverse scenarios. We theoretically prove that the hybrid loss function satisfies the properties of a strictly proper scoring rule, ensuring alignment with the true data distribution, and establish generalization error bounds, demonstrating that the model's empirical performance closely aligns with its expected performance on unseen data. Extensive experiments on both synthetic and real world datasets demonstrate the superiority of NE-GMM in terms of both predictive accuracy and uncertainty quantification.

翻译：量化预测不确定性对于现实世界的机器学习应用至关重要，尤其是在需要可靠且可解释预测的场景中。许多常见的参数化方法依赖于神经网络，通过优化负对数似然来估计分布参数。然而，这些方法常常面临训练不稳定和模式坍塌等挑战，导致对目标输出分布的均值和方差的估计不佳。在这项工作中，我们提出了神经能量高斯混合模型（NE-GMM），这是一种新颖的框架，它将高斯混合模型（GMM）与能量评分（ES）相结合，以增强预测不确定性量化。NE-GMM利用GMM的灵活性来捕捉复杂的多模态分布，并利用ES的鲁棒性来确保在不同场景下得到良好校准的预测。我们从理论上证明了该混合损失函数满足严格适当评分规则的性质，确保其与真实数据分布一致，并建立了泛化误差界，表明模型在经验上的性能与其在未见数据上的预期性能紧密一致。在合成和真实世界数据集上的大量实验证明了NE-GMM在预测准确性和不确定性量化方面的优越性。