Performance of ordinary least squares(OLS) method for the \emph{estimation of high dimensional stable state transition matrix} $A$(i.e., spectral radius $\rho(A)<1$) from a single noisy observed trajectory of the linear time invariant(LTI)\footnote{Linear Gaussian (LG) in Markov chain literature} system $X_{-}:(x_0,x_1, \ldots,x_{N-1})$ satisfying \begin{equation} x_{t+1}=Ax_{t}+w_{t}, \hspace{10pt} \text{ where } w_{t} \thicksim N(0,I_{n}), \end{equation} heavily rely on negative moments of the sample covariance matrix: $(X_{-}X_{-}^{*})=\sum_{i=0}^{N-1}x_{i}x_{i}^{*}$ and singular values of $EX_{-}^{*}$, where $E$ is a rectangular Gaussian ensemble $E=[w_0, \ldots, w_{N-1}]$. Negative moments requires sharp estimates on all the eigenvalues $\lambda_{1}\big(X_{-}X_{-}^{*}\big) \geq \ldots \geq \lambda_{n}\big(X_{-}X_{-}^{*}\big) \geq 0$. Leveraging upon recent results on spectral theorem for non-Hermitian operators in \cite{naeem2023spectral}, along with concentration of measure phenomenon and perturbation theory(Gershgorins' and Cauchys' interlacing theorem) we show that only when $A=A^{*}$, typical order of $\lambda_{j}\big(X_{-}X_{-}^{*}\big) \in \big[N-n\sqrt{N}, N+n\sqrt{N}\big]$ for all $j \in [n]$. However, in \emph{high dimensions} when $A$ has only one distinct eigenvalue $\lambda$ with geometric multiplicity of one, then as soon as eigenvalue leaves \emph{complex half unit disc}, largest eigenvalue suffers from curse of dimensionality: $\lambda_{1}\big(X_{-}X_{-}^{*}\big)=\Omega\big( \lfloor\frac{N}{n}\rfloor e^{\alpha_{\lambda}n} \big)$, while smallest eigenvalue $\lambda_{n}\big(X_{-}X_{-}^{*}\big) \in (0, N+\sqrt{N}]$. Consequently, OLS estimator incurs a \emph{phase transition} and becomes \emph{transient: increasing iteration only worsens estimation error}, all of this happening when the dynamics are generated from stable systems.

翻译：对于通过单次含噪观测轨迹估计高维稳定状态转移矩阵$A$（即谱半径$\rho(A)<1$）的普通最小二乘（OLS）方法，其性能严重依赖于样本协方差矩阵$(X_{-}X_{-}^{*})=\sum_{i=0}^{N-1}x_{i}x_{i}^{*}$与矩形高斯系综$E=[w_0,\ldots,w_{N-1}]$的奇异值$EX_{-}^{*}$的负矩。其中负矩需要所有特征值$\lambda_{1}\big(X_{-}X_{-}^{*}\big)\geq\ldots\geq\lambda_{n}\big(X_{-}X_{-}^{*}\big)\geq0$的精确估计。本文基于非厄密算子谱定理的最新成果（文献\cite{naeem2023spectral}），结合测度集中现象与微扰理论（Gershgorin圆盘定理与Cauchy交错定理），证明仅当$A=A^{*}$时，所有$j\in[n]$对应的$\lambda_{j}\big(X_{-}X_{-}^{*}\big)$的典型量级均落在区间$[N-n\sqrt{N},N+n\sqrt{N}]$内。然而在高维情形中，当$A$仅具有几何重数为一的单一特征值$\lambda$时，一旦该特征值离开复半单位圆盘，最大特征值将遭受维数灾难：$\lambda_{1}\big(X_{-}X_{-}^{*}\big)=\Omega\big( \lfloor\frac{N}{n}\rfloor e^{\alpha_{\lambda}n} \big)$，而最小特征值$\lambda_{n}\big(X_{-}X_{-}^{*}\big)\in(0,N+\sqrt{N}]$。由此导致OLS估计器出现相变并呈现瞬态特性——增加迭代次数反而恶化估计误差，而这一切均发生在动态系统由稳定系统生成的过程中。