We study the spectral properties of a class of random matrices of the form $S_n^{-} = n^{-1}(X_1 X_2^* - X_2 X_1^*)$ where $X_k = \Sigma^{1/2}Z_k$, for $k=1,2$, $Z_k$'s are independent $p\times n$ complex-valued random matrices, and $\Sigma$ is a $p\times p$ positive semi-definite matrix, independent of the $Z_k$'s. We assume that $Z_k$'s have independent entries with zero mean and unit variance. The skew-symmetric/skew-Hermitian matrix $S_n^{-}$ will be referred to as a random commutator matrix associated with the samples $X_1$ and $X_2$. We show that, when the dimension $p$ and sample size $n$ increase simultaneously, so that $p/n \to c \in (0,\infty)$, there exists a limiting spectral distribution (LSD) for $S_n^{-}$, supported on the imaginary axis, under the assumptions that the spectral distribution of $\Sigma$ converges weakly and the entries of $Z_k$'s have moments of sufficiently high order. This nonrandom LSD can be described through its Stieltjes transform, which satisfies a coupled Mar\v{c}enko-Pastur-type functional equations. In the special case when $\Sigma = I_p$, we show that the LSD of $S_n^{-}$ is a mixture of a degenerate distribution at zero (with positive mass if $c > 2$), and a continuous distribution with a symmetric density function supported on a compact interval on the imaginary axis. Moreover, we show that the companion matrix $S_n^{+} = \Sigma_n^\frac{1}{2}(Z_1Z_2^* + Z_2Z_1^*)\Sigma_n^\frac{1}{2}$, under identical assumptions, has an LSD supported on the real line, which can be similarly characterized.

翻译：我们研究一类随机矩阵 $S_n^{-} = n^{-1}(X_1 X_2^* - X_2 X_1^*)$ 的谱性质，其中 $X_k = \Sigma^{1/2}Z_k$（$k=1,2$），$Z_k$ 是独立的 $p\times n$ 复值随机矩阵，$\Sigma$ 是 $p\times p$ 半正定矩阵且独立于 $Z_k$。假设 $Z_k$ 的条目独立、零均值且具有单位方差。该斜对称/斜厄米特矩阵 $S_n^{-}$ 可视为样本 $X_1$ 与 $X_2$ 对应的随机交换子矩阵。我们证明：当维数 $p$ 与样本量 $n$ 以 $p/n \to c \in (0,\infty)$ 的方式同时增长时，在 $\Sigma$ 的谱分布弱收敛且 $Z_k$ 的条目具有足够高阶矩的假设下，$S_n^{-}$ 存在一个支撑于虚轴上的极限谱分布（LSD）。该非随机 LSD 可通过其 Stieltjes 变换进行刻画，该变换满足一组耦合的 Mar\v{c}enko-Pastur 型函数方程。特别地，当 $\Sigma = I_p$ 时，我们证明 $S_n^{-}$ 的 LSD 是零点处的退化分布（若 $c > 2$ 则具有正质量）与支撑于虚轴紧区间上、具有对称密度函数的连续分布的混合。此外，我们证明在相同假设下，伴随矩阵 $S_n^{+} = \Sigma_n^\frac{1}{2}(Z_1Z_2^* + Z_2Z_1^*)\Sigma_n^\frac{1}{2}$ 存在一个支撑于实轴上的 LSD，该 LSD 可通过类似方式刻画。