Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. In this paper, we propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR), which not only performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct. Different from typical variational autoencoders assuming I.I.D. representations (a data point's representation is not directly affected by other data points), our VIR borrows data with similar regression labels to compute the latent representation's variational distribution; furthermore, different from deterministic regression models producing point estimates, VIR predicts the entire normal-inverse-gamma distributions and modulates the associated conjugate distributions to impose probabilistic reweighting on the imbalanced data, thereby providing better uncertainty estimation. Experiments in several real-world datasets show that our VIR can outperform state-of-the-art imbalanced regression models in terms of both accuracy and uncertainty estimation. Code will soon be available at https://github.com/Wang-ML-Lab/variational-imbalanced-regression.

翻译：现有回归模型在标签分布不平衡时，往往在准确性和不确定性估计方面表现不足。本文提出了一种概率深度学习模型——变分不平衡回归（VIR），该模型不仅在不平衡回归中表现优异，还能自然地产生合理的不确定性估计作为副产品。与假设表示独立同分布（即数据点的表示不直接受其他数据点影响）的典型变分自编码器不同，我们的VIR通过借用具有相似回归标签的数据来计算潜在表示的变分分布；此外，不同于产生点估计的确定性回归模型，VIR预测完整的正态-逆伽马分布，并调节相关的共轭分布以对不平衡数据施加概率重加权，从而提供更好的不确定性估计。在多个真实世界数据集上的实验表明，我们的VIR在准确性和不确定性估计方面均能超越最先进的不平衡回归模型。代码将很快在https://github.com/Wang-ML-Lab/variational-imbalanced-regression上开源。