Noise-contrastive estimation (NCE) is a popular method for estimating unnormalised probabilistic models, such as energy-based models, which are effective for modelling complex data distributions. Unlike classical maximum likelihood (ML) estimation that relies on importance sampling (resulting in ML-IS) or MCMC (resulting in contrastive divergence, CD), NCE uses a proxy criterion to avoid the need for evaluating an often intractable normalisation constant. Despite apparent conceptual differences, we show that two NCE criteria, ranking NCE (RNCE) and conditional NCE (CNCE), can be viewed as ML estimation methods. Specifically, RNCE is equivalent to ML estimation combined with conditional importance sampling, and both RNCE and CNCE are special cases of CD. These findings bridge the gap between the two method classes and allow us to apply techniques from the ML-IS and CD literature to NCE, offering several advantageous extensions.

翻译：噪声对比估计（NCE）是一种常用于估计非归一化概率模型（如能量基模型）的流行方法，该类模型在建模复杂数据分布时具有显著效果。与经典的最大似然（ML）估计不同——后者依赖重要性抽样（产生ML-IS）或马尔可夫链蒙特卡洛方法（产生对比散度，CD），NCE采用代理准则以避免计算通常难以处理的归一化常数。尽管存在明显的概念差异，我们证明两种NCE准则——排序NCE（RNCE）和条件NCE（CNCE）——可视为ML估计方法。具体而言，RNCE等价于结合条件重要性抽样的ML估计，而RNCE和CNCE均为CD的特例。这些发现弥合了两类方法之间的鸿沟，使我们能够将ML-IS和CD文献中的技术应用于NCE，从而提供若干有利的扩展。