Generative models have been successfully used for generating realistic signals. Because the likelihood function is typically intractable in most of these models, the common practice is to use "implicit" models that avoid likelihood calculation. However, it is hard to obtain theoretical guarantees for such models. In particular, it is not understood when they can globally optimize their non-convex objectives. Here we provide such an analysis for the case of Maximum Mean Discrepancy (MMD) learning of generative models. We prove several optimality results, including for a Gaussian distribution with low rank covariance (where likelihood is inapplicable) and a mixture of Gaussians. Our analysis shows that that the MMD optimization landscape is benign in these cases, and therefore gradient based methods will globally minimize the MMD objective.

翻译：生成模型已被成功用于生成逼真的信号。由于这些模型中大多数情况下似然函数通常难以处理，常见做法是使用避免似然计算的"隐式"模型。然而，这类模型难以获得理论保证，尤其当它们需要全局优化非凸目标函数时，其优化行为尚不明确。本文针对最大均值差异（MMD）学习的生成模型提供了此类分析。我们证明了若干最优性结果，包括低秩协方差高斯分布（此时似然不适用）和高斯混合模型。分析表明，在这些情况下MMD优化景观具有良好的结构特性，因此基于梯度的优化方法能够全局最小化MMD目标函数。