I propose a novel approach for nonlinear Logistic regression using a two-layer neural network (NN) model structure with hierarchical priors on the network weights. I present a hybrid of expectation propagation called Variational Expectation Propagation approach (VEP) for approximate integration over the posterior distribution of the weights, the hierarchical scale parameters of the priors and zeta. Using a factorized posterior approximation I derive a computationally efficient algorithm, whose complexity scales similarly to an ensemble of independent sparse logistic models. The approach can be extended beyond standard activation functions and NN model structures to form flexible nonlinear binary predictors from multiple sparse linear models. I consider a hierarchical Bayesian model with logistic regression likelihood and a Gaussian prior distribution over the parameters called weights and hyperparameters. I work in the perspective of E step and M step for computing the approximating posterior and updating the parameters using the computed posterior respectively.

翻译：本文提出一种基于两层神经网络（NN）模型结构的非线性逻辑回归新方法，该结构对网络权重引入分层先验。我提出一种混合期望传播方法——变分期望传播（VEP），用于对权重后验分布、先验的分层尺度参数以及zeta进行近似积分。通过使用因子化后验近似，推导出一种计算高效的算法，其复杂度与独立稀疏逻辑回归模型的集成方法相当。该方法可扩展至标准激活函数和神经网络模型结构之外，通过多个稀疏线性模型构建灵活的非线性二元预测器。本文考虑一个分层贝叶斯模型，包含逻辑回归似然函数以及作用于参数（称为权重和超参数）上的高斯先验分布。我基于E步和M步的视角，分别计算近似后验分布并利用所得后验更新参数。