We investigate the approximation of high-dimensional target measures as low-dimensional updates of a dominating reference measure. This approximation class replaces the associated density with the composition of: (i) a feature map that identifies the leading principal components or features of the target measure, relative to the reference, and (ii) a low-dimensional profile function. When the reference measure satisfies a subspace $\phi$-Sobolev inequality, we construct a computationally tractable approximation that yields certifiable error guarantees with respect to the Amari $\alpha$-divergences. Our construction proceeds in two stages. First, for any feature map and any $\alpha$-divergence, we obtain an analytical expression for the optimal profile function. Second, for linear feature maps, the principal features are obtained from eigenvectors of a matrix involving gradients of the log-density. Neither step requires explicit access to normalizing constants. Notably, by leveraging the $\phi$-Sobolev inequalities, we demonstrate that these features universally certify approximation errors across the range of $\alpha$-divergences $\alpha \in (0,1]$. We then propose an application to Bayesian inverse problems and provide an analogous construction with approximation guarantees that hold in expectation over the data. We conclude with an extension of the proposed dimension reduction strategy to nonlinear feature maps.
翻译:我们研究将高维目标测度近似为主控参考测度的低维更新。该近似类将关联密度分解为以下两个函数的复合:(i)识别目标测度相对于参考测度的前导主成分或特征的特征映射,以及(ii)低维轮廓函数。当参考测度满足子空间Φ-Sobolev不等式时,我们构造了一个计算上可处理的近似方法,该方法针对Amari α-散度提供了可证明的误差保证。我们的构造分两个阶段进行。首先,对于任意特征映射和任意α-散度,我们获得了最优轮廓函数的解析表达式。其次,对于线性特征映射,主特征通过对数密度梯度所涉及矩阵的特征向量获得。这两个步骤均无需显式访问归一化常数。值得注意的是,通过利用Φ-Sobolev不等式,我们证明了这些特征在α-散度α∈(0,1]范围内可通用地保证近似误差。随后,我们提出了在贝叶斯逆问题中的应用,并提供了在数据期望意义上具有近似保证的类似构造。最后,我们将所提出的降维策略扩展至非线性特征映射。