We explore the probabilistic partition of unity network (PPOU-Net) model in the context of high-dimensional regression problems and propose a general framework focusing on adaptive dimensionality reduction. With the proposed framework, the target function is approximated by a mixture of experts model on a low-dimensional manifold, where each cluster is associated with a local fixed-degree polynomial. We present a training strategy that leverages the expectation maximization (EM) algorithm. During the training, we alternate between (i) applying gradient descent to update the DNN coefficients; and (ii) using closed-form formulae derived from the EM algorithm to update the mixture of experts model parameters. Under the probabilistic formulation, step (ii) admits the form of embarrassingly parallelizable weighted least-squares solves. The PPOU-Nets consistently outperform the baseline fully-connected neural networks of comparable sizes in numerical experiments of various data dimensions. We also explore the proposed model in applications of quantum computing, where the PPOU-Nets act as surrogate models for cost landscapes associated with variational quantum circuits.

翻译：我们针对高维回归问题探讨概率分割统一网络（PPOU-Net）模型，并提出一个聚焦于自适应降维的通用框架。在该框架下，目标函数通过低维流形上的专家混合模型进行逼近，其中每个聚类对应一个局部固定阶次多项式。我们提出一种利用期望最大化（EM）算法的训练策略。在训练过程中，我们交替执行：(i) 应用梯度下降更新深度神经网络（DNN）系数；(ii) 使用从EM算法导出的闭式公式更新专家混合模型参数。在概率框架下，步骤(ii)可转化为易于并行化实现的加权最小二乘求解形式。在不同数据维度的数值实验中，PPOU-Nets始终优于同等规模的基线全连接神经网络。此外，我们探索了该模型在量子计算中的应用，使PPOU-Nets作为变分量子电路相关代价地形的代理模型。