Sparse Bayesian learning (SBL) has been extensively utilized in data-driven modeling to combat the issue of overfitting. While SBL excels in linear-in-parameter models, its direct applicability is limited in models where observations possess nonlinear relationships with unknown parameters. Recently, a semi-analytical Bayesian framework known as nonlinear sparse Bayesian learning (NSBL) was introduced by the authors to induce sparsity among model parameters during the Bayesian inversion of nonlinear-in-parameter models. NSBL relies on optimally selecting the hyperparameters of sparsity-inducing Gaussian priors. It is inherently an approximate method since the uncertainty in the hyperparameter posterior is disregarded as we instead seek the maximum a posteriori (MAP) estimate of the hyperparameters (type-II MAP estimate). This paper aims to investigate the hierarchical structure that forms the basis of NSBL and validate its accuracy through a comparison with a one-level hierarchical Bayesian inference as a benchmark in the context of three numerical experiments: (i) a benchmark linear regression example with Gaussian prior and Gaussian likelihood, (ii) the same regression problem with a highly non-Gaussian prior, and (iii) an example of a dynamical system with a non-Gaussian prior and a highly non-Gaussian likelihood function, to explore the performance of the algorithm in these new settings. Through these numerical examples, it can be shown that NSBL is well-suited for physics-based models as it can be readily applied to models with non-Gaussian prior distributions and non-Gaussian likelihood functions. Moreover, we illustrate the accuracy of the NSBL algorithm as an approximation to the one-level hierarchical Bayesian inference and its ability to reduce the computational cost while adequately exploring the parameter posteriors.

翻译：稀疏贝叶斯学习（SBL）在数据驱动建模中被广泛用于克服过拟合问题。虽然SBL在参数线性模型中表现优异，但当观测值与未知参数存在非线性关系时，其直接适用性受到限制。近期，作者提出了一种名为非线性稀疏贝叶斯学习（NSBL）的半解析贝叶斯框架，该框架能在非线性参数模型的贝叶斯反演过程中诱导模型参数的稀疏性。NSBL依赖于对稀疏诱导高斯先验的超参数进行最优选择。由于我们转而寻求超参数的最大后验（MAP）估计（即第二类MAP估计），超参数后验的不确定性被忽略，因此该方法本质上是一种近似方法。本文旨在探究构成NSBL基础的层次化结构，并通过与作为基准的单层层次贝叶斯推断进行对比来验证其准确性，具体通过三项数值实验展开：（i）采用高斯先验和高斯似然函数的基准线性回归示例，（ii）同一回归问题中采用高度非高斯先验的案例，以及（iii）包含非高斯先验和高度非高斯似然函数的动力系统示例，以探索算法在这些新场景下的性能。通过这些数值示例可以证明，NSBL非常适合基于物理的模型，因为它能直接应用于具有非高斯先验分布和非高斯似然函数的模型。此外，我们展示了NSBL算法作为单层层次贝叶斯推断近似方法的准确性，以及其在充分探索参数后验分布的同时降低计算成本的能力。