Feedforward neural networks (FNNs) can be viewed as non-linear regression models, where covariates enter the model through a combination of weighted summations and non-linear functions. Although these models have some similarities to the models typically used in statistical modelling, the majority of neural network research has been conducted outside of the field of statistics. This has resulted in a lack of statistically-based methodology, and, in particular, there has been little emphasis on model parsimony. Determining the input layer structure is analogous to variable selection, while the structure for the hidden layer relates to model complexity. In practice, neural network model selection is often carried out by comparing models using out-of-sample performance. However, in contrast, the construction of an associated likelihood function opens the door to information-criteria-based variable and architecture selection. A novel model selection method, which performs both input- and hidden-node selection, is proposed using the Bayesian information criterion (BIC) for FNNs. The choice of BIC over out-of-sample performance as the model selection objective function leads to an increased probability of recovering the true model, while parsimoniously achieving favourable out-of-sample performance. Simulation studies are used to evaluate and justify the proposed method, and applications on real data are investigated.

翻译：前馈神经网络可视为非线性回归模型，其中协变量通过加权求和与非线性函数的组合进入模型。尽管这类模型与传统统计建模中使用的模型具有某些相似性，但大多数神经网络研究均在统计学领域之外进行。这导致缺乏基于统计的方法论，特别是对模型简约性的关注不足。输入层结构的确定类似于变量选择，而隐藏层结构则关乎模型复杂度。实际应用中，神经网络模型选择常通过比较样本外性能来实现。然而，与之相反，构建关联似然函数为基于信息准则的变量与架构选择开辟了道路。本文提出一种新型模型选择方法，可同时进行输入节点与隐藏节点选择，该方法基于前馈神经网络的贝叶斯信息准则。相较于将样本外性能作为模型选择目标函数，采用贝叶斯信息准则能够提高恢复真实模型的概率，同时以简约方式获得优异的样本外性能。通过仿真研究评估并验证所提方法，并在真实数据应用中进行探讨。