Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. It can be tackled with multiple hypotheses frameworks but with the difficulty of combining them efficiently in a learning model. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. The predictors are regression models of any type that can form centroidal Voronoi tessellations which are a function of their losses during training. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution and is equivalent to interpolating the meta-loss of the predictors, the loss being a zero set of the interpolation error. This model has a fixed-point iteration algorithm between the predictors and the centers of the basis functions. Diversity in learning can be controlled parametrically by truncating the tessellation formation with the losses of individual predictors. A closed-form solution with least-squares is presented, which to the authors knowledge, is the fastest solution in the literature for multiple hypotheses and structured predictions. Superior generalization performance and computational efficiency is achieved using only two-layer neural networks as predictors controlling diversity as a key component of success. A gradient-descent approach is introduced which is loss-agnostic regarding the predictors. The expected value for the loss of the structured model with Gaussian basis functions is computed, finding that correlation between predictors is not an appropriate tool for diversification. The experiments show outperformance with respect to the top competitors in the literature.

翻译：多模态回归在预测非平稳过程或具有复杂混合分布的场景中至关重要。该问题可通过多重假设框架处理，但难点在于如何在学习模型中高效整合这些假设。本文提出一种结构化径向基函数网络，作为回归问题的多假设预测器集成模型。这些预测器可为任意类型的回归模型，其能形成以训练过程中损失函数为度量的质心沃罗诺伊镶嵌。理论证明，该结构化模型能高效插值该镶嵌结构并逼近多假设目标分布，且等价于对预测器元损失进行插值——其中损失函数为插值误差的零集。该模型在预测器与基函数中心之间建立定点迭代算法，可通过截断基于单个预测器损失形成的镶嵌结构来参数化控制学习多样性。本文提出基于最小二乘法的闭式解，据作者所知，这是文献中针对多假设与结构化预测的最快解决方案。仅使用两层神经网络作为预测器即可实现优越的泛化性能与计算效率，其中多样性控制是成功的关键要素。本文还引入了一种与预测器损失函数无关的梯度下降方法，并计算了采用高斯基函数的结构化模型损失期望值，发现预测器之间的相关性并非合适的多样化工具。实验表明，该方法在性能上优于文献中的顶尖竞争者。