Mean Opinion Score (MOS) prediction has made significant progress in specific domains. However, the unstable performance of MOS prediction models across diverse samples presents ongoing challenges in the practical application of these systems. In this paper, we point out that the absence of uncertainty modeling is a significant limitation hindering MOS prediction systems from applying to the real and open world. We analyze the sources of uncertainty in the MOS prediction task and propose to establish an uncertainty-aware MOS prediction system that models aleatory uncertainty and epistemic uncertainty by heteroscedastic regression and Monte Carlo dropout separately. The experimental results show that the system captures uncertainty well and is capable of performing selective prediction and out-of-domain detection. Such capabilities significantly enhance the practical utility of MOS systems in diverse real and open-world environments.

翻译：平均意见分数预测已在特定领域取得显著进展。然而，MOS预测模型在不同样本间性能不稳定的问题，在实际系统应用中仍存在持续挑战。本文指出，不确定性建模的缺失是阻碍MOS预测系统应用于真实开放世界的重要限制因素。我们分析了MOS预测任务中不确定性的来源，提出通过异方差回归和蒙特卡洛dropout分别建模偶然不确定性和认知不确定性，从而建立不确定性感知的MOS预测系统。实验结果表明，该系统能有效捕捉不确定性，并具备选择性预测和域外检测能力。这些能力显著增强了MOS系统在多样化真实开放环境中的实际应用价值。