The Generative Adversarial Network (GAN) was recently introduced in the literature as a novel machine learning method for training generative models. It has many applications in statistics such as nonparametric clustering and nonparametric conditional independence tests. However, training the GAN is notoriously difficult due to the issue of mode collapse, which refers to the lack of diversity among generated data. In this paper, we identify the reasons why the GAN suffers from this issue, and to address it, we propose a new formulation for the GAN based on randomized decision rules. In the new formulation, the discriminator converges to a fixed point while the generator converges to a distribution at the Nash equilibrium. We propose to train the GAN by an empirical Bayes-like method by treating the discriminator as a hyper-parameter of the posterior distribution of the generator. Specifically, we simulate generators from its posterior distribution conditioned on the discriminator using a stochastic gradient Markov chain Monte Carlo (MCMC) algorithm, and update the discriminator using stochastic gradient descent along with simulations of the generators. We establish convergence of the proposed method to the Nash equilibrium. Apart from image generation, we apply the proposed method to nonparametric clustering and nonparametric conditional independence tests. A portion of the numerical results is presented in the supplementary material.

翻译：生成对抗网络（GAN）近期在文献中被提出作为一种训练生成模型的创新机器学习方法。它在统计学中具有诸多应用，例如非参数聚类和非参数条件独立性检验。然而，由于模式崩溃问题（即生成数据缺乏多样性），训练GAN极其困难。本文揭示了GAN遭遇该问题的根本原因，并为此提出了一种基于随机决策规则的GAN新公式。在新公式中，判别器收敛至固定点，而生成器在纳什均衡处收敛为分布。我们提出通过经验贝叶斯类方法训练GAN，将判别器视为生成器后验分布的超参数。具体而言，我们利用随机梯度马尔可夫链蒙特卡洛（MCMC）算法从给定判别器的生成器后验分布中采样生成器，并沿生成器模拟过程使用随机梯度下降更新判别器。我们证明了所提方法收敛至纳什均衡。除图像生成外，我们将该方法应用于非参数聚类和非参数条件独立性检验。部分数值结果见补充材料。