Bayes' rule describes how to infer posterior beliefs about latent variables given observations, and inference is a critical step in learning algorithms for latent variable models (LVMs). Although there are exact algorithms for inference and learning for certain LVMs such as linear Gaussian models and mixture models, researchers must typically develop approximate inference and learning algorithms when applying novel LVMs. In this paper we study the line that separates LVMs that rely on approximation schemes from those that do not, and develop a general theory of exponential family, latent variable models for which inference and learning may be implemented exactly. Firstly, under mild assumptions about the exponential family form of a given LVM, we derive necessary and sufficient conditions under which the LVM prior is in the same exponential family as its posterior, such that the prior is conjugate to the posterior. We show that all models that satisfy these conditions are constrained forms of a particular class of exponential family graphical model. We then derive general inference and learning algorithms, and demonstrate them on a variety of example models. Finally, we show how to compose our models into graphical models that retain tractable inference and learning. In addition to our theoretical work, we have implemented our algorithms in a collection of libraries with which we provide numerous demonstrations of our theory, and with which researchers may apply our theory in novel statistical settings.

翻译：贝叶斯法则描述了在给定观测数据下如何推断潜变量的后验信念，而推理是潜变量模型（LVM）学习算法中的关键步骤。尽管针对线性高斯模型和混合模型等特定LVM存在精确的推理与学习算法，但在应用新型LVM时，研究者通常需要开发近似推理与学习算法。本文研究了区分需采用近似方案与无需近似方案的LVM的分界线，并建立了可精确实现推理与学习的指数族潜变量模型的一般理论。首先，在给定LVM指数族形式的温和假设下，我们推导了该LVM先验分布与其后验分布同属同一指数族的充分必要条件，使得先验与后验共轭。研究表明，满足这些条件的所有模型均为特定指数族图模型的约束形式。随后，我们推导了通用的推理与学习算法，并在多种示例模型上进行了验证。最后，我们展示了如何将所提模型组合为仍保持可处理推理与学习的图模型。除理论工作外，我们已在一系列库中实现了相关算法，并提供了大量理论演示示例，研究者可借此在新型统计场景中应用我们的理论。