The fight between discriminative versus generative goes deep, in both the study of artificial and natural intelligence. In our view, both camps have complementary values. So, we sought to synergistically combine them. Here, we propose a methodology to convert deep discriminative networks to kernel generative networks. We leveraged the fact that deep models, including both random forests and deep networks, learn internal representations which are unions of polytopes with affine activation functions to conceptualize them both as generalized partitioning rules. We replace the affine function in each polytope populated by the training data with Gaussian kernel that results in a generative model. Theoretically, we derive the conditions under which our generative models are a consistent estimator of the corresponding class conditional density. Moreover, our proposed models obtain well calibrated posteriors for in-distribution, and extrapolate beyond the training data to handle out-of-distribution inputs reasonably. We believe this approach may be an important step in unifying the thinking and the approaches across the discriminative and the generative divide.

翻译：判别式与生成式之间的争论在人工和自然智能研究中根深蒂固。我们认为，这两个阵营具有互补价值，因此寻求将其协同融合。本文提出了一种将深度判别网络转换为核生成网络的方法。我们利用深度模型（包括随机森林和深度网络）学习内部表征的特点——这些表征是具有仿射激活函数的多面体并集——将其概念化为广义划分规则。我们用高斯核替换每个由训练数据填充的多面体中的仿射函数，从而形成生成模型。理论上，我们推导了该生成模型作为对应类条件密度一致估计量的条件。此外，我们提出的模型能够获得分布内数据的高度校准后验概率，并在训练数据之外合理外推处理分布外输入。我们认为，这一方法可能是统一判别式与生成式鸿沟思维与路径的重要一步。