We present the new Orthogonal Polynomials Approximation Algorithm (OPAA), a parallelizable algorithm that solves two problems from a functional analytic approach: first, it finds a smooth functional estimate of a density function, whether it is normalized or not; second, the algorithm provides an estimate of the normalizing weight. In the context of Bayesian inference, OPAA provides an estimate of the posterior function as well as the normalizing weight, which is also known as the evidence. A core component of OPAA is a special transform of the square root of the joint distribution into a special functional space of our construct. Through this transform, the evidence is equated with the $L^2$ norm of the transformed function, squared. Hence, the evidence can be estimated by the sum of squares of the transform coefficients. The computations can be parallelized and completed in one pass. To compute the transform coefficients, OPAA proposes a new computational scheme leveraging Gauss--Hermite quadrature in higher dimensions. Not only does it avoid the potential high variance problem associated with random sampling methods, it also enables one to speed up the computation by parallelization, and significantly reduces the complexity by a vector decomposition.
翻译:我们提出了新的正交多项式逼近算法(OPAA),这是一种可并行化的算法,通过泛函分析方法解决两个问题:第一,对密度函数(无论是否归一化)给出光滑的泛函估计;第二,该算法提供归一化权重的估计。在贝叶斯推断背景下,OPAA既能给出后验函数的估计,也能给出归一化权重(即证据)的估计。OPAA的核心组成部分是将联合分布的平方根变换到我们构建的特殊函数空间。通过该变换,证据等于变换后函数的$L^2$范数的平方,因此可通过变换系数平方和进行估计。所有计算可并行化并一次性完成。为计算变换系数,OPAA提出了利用高维Gauss-Hermite求积的新计算方案。该方法不仅避免了随机采样方法可能带来的高方差问题,还能通过并行化加速计算,并通过向量分解显著降低计算复杂度。