Normalizing Flows (NFs) are powerful and efficient models for density estimation. When modeling densities on manifolds, NFs can be generalized to injective flows but the Jacobian determinant becomes computationally prohibitive. Current approaches either consider bounds on the log-likelihood or rely on some approximations of the Jacobian determinant. In contrast, we propose injective flows for star-like manifolds and show that for such manifolds we can compute the Jacobian determinant exactly and efficiently, with the same cost as NFs. This aspect is particularly relevant for variational inference settings, where no samples are available and only some unnormalized target is known. Among many, we showcase the relevance of modeling densities on star-like manifolds in two settings. Firstly, we introduce a novel Objective Bayesian approach for penalized likelihood models by interpreting level-sets of the penalty as star-like manifolds. Secondly, we consider probabilistic mixing models and introduce a general method for variational inference by defining the posterior of mixture weights on the probability simplex.
翻译:归一化流(NFs)是用于密度估计的强大高效模型。在对流形上的密度进行建模时,NFs可以推广为单射流,但此时雅可比行列式的计算变得难以处理。现有方法要么考虑对数似然的边界,要么依赖于对雅可比行列式的某种近似。相比之下,我们提出了针对星状流形的单射流,并证明对于此类流形,我们可以精确且高效地计算雅可比行列式,其计算成本与NFs相同。这一特性在变分推断场景中尤为重要,因为此类场景中通常没有可用样本,仅已知某个未归一化的目标分布。我们通过多种案例展示了在星状流形上建模密度的相关性,并重点介绍了两种应用场景。首先,我们通过将惩罚项的水平集解释为星状流形,为惩罚似然模型提出了一种新颖的客观贝叶斯方法。其次,我们考虑概率混合模型,通过在概率单纯形上定义混合权重的后验分布,提出了一种通用的变分推断方法。