Linear discriminant analysis (LDA) has been a useful tool in pattern recognition and data analysis research and practice. While linearity of class boundaries cannot always be expected, nonlinear projections through pre-trained deep neural networks have served to map complex data onto feature spaces in which linear discrimination has served well. The solution to binary LDA is obtained by eigenvalue analysis of within-class and between-class scatter matrices. It is well known that the multiclass LDA is solved by an extension to the binary LDA, a generalised eigenvalue problem, from which the largest subspace that can be extracted is of dimension one lower than the number of classes in the given problem. In this paper, we show that, apart from the first of the discriminant directions, the generalised eigenanalysis solution to multiclass LDA does neither yield orthogonal discriminant directions nor maximise discrimination of projected data along them. Surprisingly, to the best of our knowledge, this has not been noted in decades of literature on LDA. To overcome this drawback, we present a derivation with a strict theoretical support for sequentially obtaining discriminant directions that are orthogonal to previously computed ones and maximise in each step the Fisher criterion. We show distributions of projections along these axes and demonstrate that discrimination of data projected onto these discriminant directions has optimal separation, which is much higher than those from the generalised eigenvectors of the multiclass LDA. Using a wide range of benchmark tasks, we present a comprehensive empirical demonstration that on a number of pattern recognition and classification problems, the optimal discriminant subspaces obtained by the proposed method, referred to as GO-LDA (Generalised Optimal LDA), can offer superior accuracy.

翻译：线性判别分析（LDA）一直是模式识别与数据分析研究及实践中行之有效的工具。虽然类别边界的线性特征并非总能得到保证，但通过预训练深度神经网络实现的非线性投影，可将复杂数据映射到便于线性判别的特征空间。二元LDA的解可通过类内散度矩阵与类间散度矩阵的特征值分析获得。众所周知，多类LDA通过将二元LDA扩展为广义特征值问题求解，其最大可提取子空间的维数比给定问题中的类别数少一。本文表明，除第一个判别方向外，多类LDA的广义特征分析解既不能产生正交的判别方向，也不能最大化投影数据沿这些方向的判别能力。令人惊讶的是，据我们所知，这一缺陷在数十年的LDA文献中未被注意到。为克服此缺陷，我们提出一种具有严格理论支持的推导方法，可顺序获取与先前计算方向正交且在每一步最大化Fisher准则的判别方向。我们展示了沿这些轴的投影分布，并证明投影到这些判别方向上的数据具有最优分离度，其分离效果远优于多类LDA的广义特征向量。通过广泛的基准任务，我们综合性地实证表明：在诸多模式识别与分类问题中，由所提方法（称为GO-LDA，即广义最优LDA）获得的最优判别子空间能够提供更优的分类准确率。