Deep learning has achieved tremendous success. \nj{However,} unlike SVMs, which provide direct decision criteria and can be trained with a small dataset, it still has significant weaknesses due to its requirement for massive datasets during training and the black-box characteristics on decision criteria. \nj{This paper addresses} these issues by identifying support vectors in deep learning models. To this end, we propose the DeepKKT condition, an adaptation of the traditional Karush-Kuhn-Tucker (KKT) condition for deep learning models, and confirm that generated Deep Support Vectors (DSVs) using this condition exhibit properties similar to traditional support vectors. This allows us to apply our method to few-shot dataset distillation problems and alleviate the black-box characteristics of deep learning models. Additionally, we demonstrate that the DeepKKT condition can transform conventional classification models into generative models with high fidelity, particularly as latent \jh{generative} models using class labels as latent variables. We validate the effectiveness of DSVs \nj{using common datasets (ImageNet, CIFAR10 \nj{and} CIFAR100) on the general architectures (ResNet and ConvNet)}, proving their practical applicability. (See Fig.~\ref{fig:generated})

翻译：深度学习已取得巨大成功。然而，与支持向量机（SVM）相比，后者能提供直接的决策标准且可通过小数据集进行训练，深度学习仍存在显著弱点，这主要源于其在训练过程中对海量数据的需求以及决策标准的黑箱特性。本文通过识别深度学习模型中的支持向量来解决这些问题。为此，我们提出了DeepKKT条件，这是对传统Karush-Kuhn-Tucker（KKT）条件在深度学习模型中的一种适应，并证实了使用该条件生成的深度支持向量（DSV）具有与传统支持向量相似的性质。这使得我们的方法能够应用于少样本数据集蒸馏问题，并缓解深度学习模型的黑箱特性。此外，我们证明了DeepKKT条件可以将传统的分类模型转化为具有高保真度的生成模型，特别是作为以类别标签作为隐变量的隐式生成模型。我们在通用架构（ResNet和ConvNet）上使用常见数据集（ImageNet、CIFAR10和CIFAR100）验证了DSV的有效性，证明了其实际适用性。（参见图~\ref{fig:generated}）