Energy-Based Models (EBMs) are an important class of probabilistic models, also known as random fields and undirected graphical models. EBMs are un-normalized and thus radically different from other popular self-normalized probabilistic models such as hidden Markov models (HMMs), autoregressive models, generative adversarial nets (GANs) and variational auto-encoders (VAEs). Over the past years, EBMs have attracted increasing interest not only from the core machine learning community, but also from application domains such as speech, vision, natural language processing (NLP) and so on, due to significant theoretical and algorithmic progress. The sequential nature of speech and language also presents special challenges and needs a different treatment from processing fix-dimensional data (e.g., images). Therefore, the purpose of this monograph is to present a systematic introduction to energy-based models, including both algorithmic progress and applications in speech and language processing. First, the basics of EBMs are introduced, including classic models, recent models parameterized by neural networks, sampling methods, and various learning methods from the classic learning algorithms to the most advanced ones. Then, the application of EBMs in three different scenarios is presented, i.e., for modeling marginal, conditional and joint distributions, respectively. 1) EBMs for sequential data with applications in language modeling, where the main focus is on the marginal distribution of a sequence itself; 2) EBMs for modeling conditional distributions of target sequences given observation sequences, with applications in speech recognition, sequence labeling and text generation; 3) EBMs for modeling joint distributions of both sequences of observations and targets, and their applications in semi-supervised learning and calibrated natural language understanding.

翻译：基于能量的模型（EBMs）是一类重要的概率模型，亦称为随机场与无向图模型。EBM属于非归一化模型，因此与隐马尔可夫模型（HMMs）、自回归模型、生成对抗网络（GANs）及变分自编码器（VAEs）等流行的自归一化概率模型有本质区别。近年来，由于理论与算法方面的显著进展，EBM不仅吸引了核心机器学习领域的广泛关注，还引起了语音、视觉、自然语言处理（NLP）等应用领域的兴趣。语音与语言的序列特性带来了特殊挑战，需要与处理固定维度数据（如图像）不同的方法。因此，本专著的目的是系统介绍基于能量的模型，涵盖算法进展及其在语音与语言处理中的应用。首先，介绍EBM的基础知识，包括经典模型、近期基于神经网络参数化的模型、采样方法以及从经典学习算法到最先进方法的各类学习方法。随后，分别阐述EBM在三种不同场景中的应用，即用于建模边际分布、条件分布与联合分布：1）面向序列数据的EBM及其在语言建模中的应用（主要关注序列本身的边际分布）；2）面向给定观测序列的目标序列条件分布建模的EBM，应用于语音识别、序列标注及文本生成；3）面向观测序列与目标序列联合分布建模的EBM，及其在半监督学习与校准型自然语言理解中的应用。