Statistical models are central to machine learning with broad applicability across a range of downstream tasks. The models are controlled by free parameters that are typically estimated from data by maximum-likelihood estimation or approximations thereof. However, when faced with real-world datasets many of the models run into a critical issue: they are formulated in terms of fully-observed data, whereas in practice the datasets are plagued with missing data. The theory of statistical model estimation from incomplete data is conceptually similar to the estimation of latent-variable models, where powerful tools such as variational inference (VI) exist. However, in contrast to standard latent-variable models, parameter estimation with incomplete data often requires estimating exponentially-many conditional distributions of the missing variables, hence making standard VI methods intractable. We address this gap by introducing variational Gibbs inference (VGI), a new general-purpose method to estimate the parameters of statistical models from incomplete data. We validate VGI on a set of synthetic and real-world estimation tasks, estimating important machine learning models such as VAEs and normalising flows from incomplete data. The proposed method, whilst general-purpose, achieves competitive or better performance than existing model-specific estimation methods.

翻译：统计模型是机器学习的核心，广泛应用于各类下游任务中。这些模型由自由参数控制，通常通过最大似然估计或其近似方法从数据中估计这些参数。然而，面对真实世界的数据集时，许多模型会遇到一个关键问题：它们基于完全观测数据构建，而实际数据集常受缺失数据困扰。从不完整数据中估计统计模型的理论在概念上与潜变量模型的估计相似，后者已有变分推断（VI）等强大工具。但与标准潜变量模型不同，不完整数据下的参数估计通常需要估计缺失变量的指数级数量的条件分布，这使得标准VI方法难以处理。我们通过引入变分吉布斯推断（VGI）来填补这一空白，这是一种从缺失数据中估计统计模型参数的新型通用方法。我们在合成与真实估计任务上验证了VGI，包括从缺失数据中估计变分自编码器（VAE）和标准化流等重要机器学习模型。所提出的方法虽为通用设计，但在性能上达到或优于现有特定模型估计方法。