Generative models have advantageous characteristics for classification tasks such as the availability of unsupervised data and calibrated confidence, whereas discriminative models have advantages in terms of the simplicity of their model structures and learning algorithms and their ability to outperform their generative counterparts. In this paper, we propose a method to train a hybrid of discriminative and generative models in a single neural network (NN), which exhibits the characteristics of both models. The key idea is the Gaussian-coupled softmax layer, which is a fully connected layer with a softmax activation function coupled with Gaussian distributions. This layer can be embedded into an NN-based classifier and allows the classifier to estimate both the class posterior distribution and the class-conditional data distribution. We demonstrate that the proposed hybrid model can be applied to semi-supervised learning and confidence calibration.

翻译：生成式模型在分类任务中具有利用无监督数据和校准置信度等优势特性，而判别式模型则以其模型结构与学习算法的简洁性以及超越生成式模型的性能见长。本文提出一种在单一神经网络中训练判别式与生成式混合模型的方法，该模型同时具备两类模型的特性。核心技术在于高斯耦合Softmax层——这是一种与高斯分布相耦合、采用Softmax激活函数的全连接层。该层可嵌入基于神经网络的分类器，使其能够同时估计类别后验分布和类别条件数据分布。我们证明，所提出的混合模型可应用于半监督学习和置信度校准。