In this paper, we aim at establishing an approximation theory and a learning theory of distribution regression via a fully connected neural network (FNN). In contrast to the classical regression methods, the input variables of distribution regression are probability measures. Then we often need to perform a second-stage sampling process to approximate the actual information of the distribution. On the other hand, the classical neural network structure requires the input variable to be a vector. When the input samples are probability distributions, the traditional deep neural network method cannot be directly used and the difficulty arises for distribution regression. A well-defined neural network structure for distribution inputs is intensively desirable. There is no mathematical model and theoretical analysis on neural network realization of distribution regression. To overcome technical difficulties and address this issue, we establish a novel fully connected neural network framework to realize an approximation theory of functionals defined on the space of Borel probability measures. Furthermore, based on the established functional approximation results, in the hypothesis space induced by the novel FNN structure with distribution inputs, almost optimal learning rates for the proposed distribution regression model up to logarithmic terms are derived via a novel two-stage error decomposition technique.

翻译：本文旨在建立基于全连接神经网络（FNN）的分布回归逼近理论与学习理论。区别于经典回归方法，分布回归的输入变量为概率测度。此时通常需进行第二阶段采样过程以近似分布的实际信息。另一方面，经典神经网络结构要求输入变量为向量形式。当输入样本为概率分布时，传统深度神经网络方法无法直接应用，从而引发分布回归的困难。对于分布输入而言，建立定义明确的神经网络结构具有迫切需求。目前尚缺乏针对分布回归的神经网络实现的数学模型与理论分析。为克服技术难点并解决该问题，我们构建了一种新型全连接神经网络框架，实现了对Borel概率测度空间上定义的泛函的逼近理论。进一步，基于所建立的泛函逼近结果，在由该新型FNN结构（含分布输入）诱导的假设空间中，通过创新的两阶段误差分解技术，推导出所提分布回归模型在忽略对数项情况下的几乎最优学习速率。