Data imbalance is ubiquitous when applying machine learning to real-world problems, particularly regression problems. If training data are imbalanced, the learning is dominated by the densely covered regions of the target distribution, consequently, the learned regressor tends to exhibit poor performance in sparsely covered regions. Beyond standard measures like over-sampling or re-weighting, there are two main directions to handle learning from imbalanced data. For regression, recent work relies on the continuity of the distribution; whereas for classification there has been a trend to employ mixture-of-expert models and let some ensemble members specialize in predictions for the sparser regions. Here, we adapt the mixture-of-experts approach to the regression setting. A main question when using this approach is how to fuse the predictions from multiple experts into one output. Drawing inspiration from recent work on probabilistic deep learning, we propose to base the fusion on the aleatoric uncertainties of individual experts, thus obviating the need for a separate aggregation module. In our method, dubbed MOUV, each expert predicts not only an output value but also its uncertainty, which in turn serves as a statistically motivated criterion to rely on the right experts. We compare our method with existing alternatives on multiple public benchmarks and show that MOUV consistently outperforms the prior art, while at the same time producing better calibrated uncertainty estimates. Our code is available at link-upon-publication.

翻译：数据不平衡在机器学习应用于实际问题时普遍存在，尤其是在回归问题中。若训练数据存在不平衡，学习过程将由目标分布中密集覆盖的区域主导，导致学习到的回归器在稀疏覆盖区域表现不佳。除过采样或重加权等标准方法外，处理不平衡数据学习主要有两个方向。针对回归问题，近期研究依赖于分布的连续性；而分类问题则倾向于采用专家混合模型，让部分集成成员专门处理稀疏区域的预测。本研究将专家混合方法适应于回归场景，其核心问题在于如何将多个专家的预测融合为单一输出。受概率深度学习最新研究的启发，我们提出基于各专家偶然不确定性进行融合，从而无需单独的聚合模块。在名为MOUV的该方法中，每个专家不仅预测输出值，还预测其不确定性，此不确定性作为统计驱动的准则来决定依赖哪些专家。我们在多个公开基准上比较该方法与现有替代方案，结果表明MOUV始终优于现有技术，同时能生成校准更优的不确定性估计。相关代码将在论文发表后公开。