Linear discriminant analysis (LDA) is a fundamental classification and dimension reduction method that achieves Bayes optimality under Gaussian mixture, but often struggles in high-dimensional settings where the covariance matrix cannot be reliably estimated. We propose LDA with gradient optimization (LDA-GO), which learns a low-rank precision matrix via scalable gradient-based optimization. The method automatically selects between a Gaussian likelihood and a cross-entropy loss using data-driven structural diagnostics, adapting to the signal structure without user tuning. The gradient computation avoids any quadratic-sized intermediate matrix, keeping the per-iteration cost linear in the number of dimensions. Theoretically, we prove several properties of the method, including the convexity of the objective functions, Bayes-optimality of the method, and a finite-sample bound of the excess error. Numerically, we conducted a variety of simulations and real data experiments to show that LDA-GO wins a majority of settings among other LDA variants, particularly in sparse-signal high-dimensional regimes.

翻译：线性判别分析（LDA）是一种基础的分类与降维方法，在高斯混合模型下可实现贝叶斯最优性，但在协方差矩阵难以可靠估计的高维场景中常面临挑战。我们提出基于梯度优化的线性判别分析（LDA-GO），该方法通过可扩展的梯度优化学习低秩精度矩阵。该方法利用数据驱动的结构诊断自动选择高斯似然或交叉熵损失，无需用户调参即可适应信号结构。梯度计算避免了任何二次规模的中间矩阵，使每次迭代的计算成本与维度呈线性关系。理论上，我们证明了该方法的若干性质，包括目标函数的凸性、方法的贝叶斯最优性以及超额误差的有限样本界。在数值实验中，我们通过多种模拟和真实数据实验表明，LDA-GO在其他LDA变体中赢得了大多数场景，尤其是在稀疏信号的高维情形中表现突出。