In the last decades, energy-based models (EBMs) have become an important class of probabilistic models in which a component of the likelihood is intractable and therefore cannot be evaluated explicitly. Consequently, parameter estimation in EBMs is challenging for conventional inference methods. In this work, we provide a unified framework that connects noise contrastive estimation (NCE), reverse logistic regression (RLR), multiple importance sampling (MIS), and bridge sampling within the context of EBMs. We further show that these methods are equivalent under specific conditions. This unified perspective clarifies relationships among existing methods and enables the development of new estimators, with the potential to improve statistical and computational efficiency. Furthermore, this study helps elucidate the success of NCE in terms of its flexibility and robustness, while also identifying scenarios in which its performance can be further improved. Hence, rather than being a purely descriptive review, this work offers a unifying perspective and additional methodological contributions. The MATLAB code used in the numerical experiments is also made freely available to support the reproducibility of the results.
翻译:在过去的几十年中,基于能量的模型(EBMs)已成为一类重要的概率模型,其似然函数中的某个组成部分难以处理,因此无法显式评估。因此,在EBM中进行参数估计对于传统推断方法来说具有挑战性。在本工作中,我们提供了一个统一框架,将噪声对比估计(NCE)、反向逻辑回归(RLR)、多重重要性采样(MIS)和桥接采样在EBM语境中联系起来。我们进一步展示了这些方法在特定条件下是等价的。这一统一视角澄清了现有方法之间的关系,并促进了新估计量的开发,具有提升统计和计算效率的潜力。此外,本研究有助于阐明NCE在灵活性和鲁棒性方面的成功,同时也识别出可以进一步提升其性能的场景。因此,本工作并非单纯的描述性综述,而是提供了一个统一视角并提出了额外的方法论贡献。数值实验所用的MATLAB代码也免费提供,以支持结果的可复现性。