Likelihood-based deep generative models (DGMs) commonly exhibit a puzzling behaviour: when trained on a relatively complex dataset, they assign higher likelihood values to out-of-distribution (OOD) data from simpler sources. Adding to the mystery, OOD samples are never generated by these DGMs despite having higher likelihoods. This two-pronged paradox has yet to be conclusively explained, making likelihood-based OOD detection unreliable. Our primary observation is that high-likelihood regions will not be generated if they contain minimal probability mass. We demonstrate how this seeming contradiction of large densities yet low probability mass can occur around data confined to low-dimensional manifolds. We also show that this scenario can be identified through local intrinsic dimension (LID) estimation, and propose a method for OOD detection which pairs the likelihoods and LID estimates obtained from a pre-trained DGM. Our method can be applied to normalizing flows and score-based diffusion models, and obtains results which match or surpass state-of-the-art OOD detection benchmarks using the same DGM backbones. Our code is available at https://github.com/layer6ai-labs/dgm_ood_detection.

翻译：基于似然性的深度生成模型（DGMs）通常表现出一种令人困惑的行为：当在相对复杂的数据集上训练时，它们会为来自更简单来源的分布外（OOD）数据分配更高的似然值。更添神秘的是，尽管具有更高的似然值，这些DGMs却从未生成过OOD样本。这个双重的悖论尚未得到确切的解释，使得基于似然性的OOD检测不可靠。我们的主要观察是，如果高似然区域包含的概率质量极小，则不会被生成。我们论证了这种看似矛盾的现象——即高密度却伴随低概率质量——如何可能发生在局限于低维流形的数据周围。我们还表明，这种情况可以通过局部本征维度（LID）估计来识别，并提出了一种OOD检测方法，该方法将来自预训练DGM的似然值与LID估计值配对使用。我们的方法可应用于标准化流和基于分数的扩散模型，并且在使用相同DGM主干网络的情况下，取得了与最先进的OOD检测基准相当或更优的结果。我们的代码可在 https://github.com/layer6ai-labs/dgm_ood_detection 获取。