Covariance and Hessian matrices have been analyzed separately in the literature for classification problems. However, integrating these matrices has the potential to enhance their combined power in improving classification performance. We present a novel approach that combines the eigenanalysis of a covariance matrix evaluated on a training set with a Hessian matrix evaluated on a deep learning model to achieve optimal class separability in binary classification tasks. Our approach is substantiated by formal proofs that establish its capability to maximize between-class mean distance and minimize within-class variances. By projecting data into the combined space of the most relevant eigendirections from both matrices, we achieve optimal class separability as per the linear discriminant analysis (LDA) criteria. Empirical validation across neural and health datasets consistently supports our theoretical framework and demonstrates that our method outperforms established methods. Our method stands out by addressing both LDA criteria, unlike PCA and the Hessian method, which predominantly emphasize one criterion each. This comprehensive approach captures intricate patterns and relationships, enhancing classification performance. Furthermore, through the utilization of both LDA criteria, our method outperforms LDA itself by leveraging higher-dimensional feature spaces, in accordance with Cover's theorem, which favors linear separability in higher dimensions. Our method also surpasses kernel-based methods and manifold learning techniques in performance. Additionally, our approach sheds light on complex DNN decision-making, rendering them comprehensible within a 2D space.

翻译：在分类问题的文献中，协方差矩阵与海森矩阵通常被单独分析。然而，整合这两类矩阵有望提升其在改善分类性能方面的综合能力。我们提出了一种新方法，将训练集上计算的协方差矩阵的特征分析与深度学习模型上计算的海森矩阵的特征分析相结合，以实现二分类任务中的最优类别可分性。该方法有正式理论证明作为支撑，证实其能够最大化类间均值距离并最小化类内方差。通过将数据投影至两个矩阵中最相关特征方向构成的联合空间，我们依据线性判别分析(LDA)准则实现了最优类别可分性。在神经与健康数据集上的实证验证一致地支持了我们的理论框架，并表明该方法优于现有方法。与主要分别侧重单一准则的主成分分析(PCA)和海森方法不同，本方法通过同时满足LDA的两项准则脱颖而出。这种综合性方法能够捕捉复杂模式与关系，从而提升分类性能。此外，通过利用LDA的两项准则，本方法依据柯弗定理中高维空间有利于线性可分性的原理，借助更高维特征空间超越了LDA本身。本方法的性能也优于基于核的方法和流形学习技术。同时，该方法揭示了深度神经网络复杂的决策机制，使其在二维空间中变得可解释。