Neural networks have shown state-of-the-art performances in various classification and regression tasks. Rectified linear units (ReLU) are often used as activation functions for the hidden layers in a neural network model. In this article, we establish the connection between the Poisson hyperplane processes (PHP) and two-layer ReLU neural networks. We show that the PHP with a Gaussian prior is an alternative probabilistic representation to a two-layer ReLU neural network. In addition, we show that a two-layer neural network constructed by PHP is scalable to large-scale problems via the decomposition propositions. Finally, we propose an annealed sequential Monte Carlo algorithm for Bayesian inference. Our numerical experiments demonstrate that our proposed method outperforms the classic two-layer ReLU neural network. The implementation of our proposed model is available at https://github.com/ShufeiGe/Pois_Relu.git.

翻译：神经网络在各类分类与回归任务中展现出最先进的性能。修正线性单元（ReLU）常被用作神经网络模型隐藏层的激活函数。本文建立了泊松超平面过程（PHP）与两层ReLU神经网络之间的联系。我们证明，具有高斯先验的PHP是两层ReLU神经网络的另一种概率表示。此外，通过分解命题，我们证明了由PHP构建的两层神经网络可扩展至大规模问题。最后，我们提出了一种用于贝叶斯推断的退火序贯蒙特卡罗算法。数值实验表明，我们提出的方法优于经典的两层ReLU神经网络。所提模型的实现代码可在 https://github.com/ShufeiGe/Pois_Relu.git 获取。