We present a supervised learning framework of training generative models for density estimation. Generative models, including generative adversarial networks, normalizing flows, variational auto-encoders, are usually considered as unsupervised learning models, because labeled data are usually unavailable for training. Despite the success of the generative models, there are several issues with the unsupervised training, e.g., requirement of reversible architectures, vanishing gradients, and training instability. To enable supervised learning in generative models, we utilize the score-based diffusion model to generate labeled data. Unlike existing diffusion models that train neural networks to learn the score function, we develop a training-free score estimation method. This approach uses mini-batch-based Monte Carlo estimators to directly approximate the score function at any spatial-temporal location in solving an ordinary differential equation (ODE), corresponding to the reverse-time stochastic differential equation (SDE). This approach can offer both high accuracy and substantial time savings in neural network training. Once the labeled data are generated, we can train a simple fully connected neural network to learn the generative model in the supervised manner. Compared with existing normalizing flow models, our method does not require to use reversible neural networks and avoids the computation of the Jacobian matrix. Compared with existing diffusion models, our method does not need to solve the reverse-time SDE to generate new samples. As a result, the sampling efficiency is significantly improved. We demonstrate the performance of our method by applying it to a set of 2D datasets as well as real data from the UCI repository.

翻译：我们提出了一种用于密度估计的生成模型监督学习框架。生成模型，包括生成对抗网络、归一化流、变分自编码器，通常被视为无监督学习模型，因为训练时通常缺乏标注数据。尽管生成模型取得了成功，但无监督训练仍存在若干问题，例如需要可逆架构、梯度消失以及训练不稳定。为了在生成模型中实现监督学习，我们利用基于得分的扩散模型生成标注数据。与现有训练神经网络学习得分函数的扩散模型不同，我们开发了一种无需训练的得分估计方法。该方法使用基于小批量蒙特卡洛估计器直接近似任意时空位置上的得分函数，以求解对应逆向时间随机微分方程的常微分方程。该方法既能提供高精度，又能显著节省神经网络训练的时间。一旦生成标注数据，我们便可以以监督方式训练一个简单的全连接神经网络来学习生成模型。与现有归一化流模型相比，我们的方法无需使用可逆神经网络，并避免了雅可比矩阵的计算。与现有扩散模型相比，我们的方法无需求解逆向时间随机微分方程即可生成新样本，从而显著提高了采样效率。我们将该方法应用于一系列二维数据集以及UCI数据库中的真实数据，展示了其性能。