The combination of Terahertz (THz) and massive multiple-input multiple-output (MIMO) is promising to meet the increasing data rate demand of future wireless communication systems thanks to the huge bandwidth and spatial degrees of freedom. However, unique channel features such as the near-field beam split effect make channel estimation particularly challenging in THz massive MIMO systems. On one hand, adopting the conventional angular domain transformation dictionary designed for low-frequency far-field channels will result in degraded channel sparsity and destroyed sparsity structure in the transformed domain. On the other hand, most existing compressive sensing-based channel estimation algorithms cannot achieve high performance and low complexity simultaneously. To alleviate these issues, in this paper, we first adopt frequency-dependent near-field dictionaries to maintain good channel sparsity and sparsity structure in the transformed domain under the near-field beam split effect. Then, a deep unfolding-based wideband THz massive MIMO channel estimation algorithm is proposed. In each iteration of the unitary approximate message passing-sparse Bayesian learning algorithm, the optimal update rule is learned by a deep neural network (DNN), whose structure is customized to effectively exploit the inherent channel patterns. Furthermore, a mixed training method based on novel designs of the DNN structure and the loss function is developed to effectively train data from different system configurations. Simulation results validate the superiority of the proposed algorithm in terms of performance, complexity, and robustness.

翻译：太赫兹（THz）与大规模多输入多输出（massive MIMO）的结合，凭借其巨大的带宽和空间自由度，有望满足未来无线通信系统日益增长的数据速率需求。然而，近场波束分裂效应等独特的信道特征使得THz大规模MIMO系统中的信道估计尤为困难。一方面，采用为低频远场信道设计的传统角度域变换字典会导致变换域中的信道稀疏性下降及稀疏结构破坏。另一方面，现有基于压缩感知的信道估计算法大多无法同时实现高性能与低复杂度。为缓解这些问题，本文首先采用频率相关的近场字典，以在近场波束分裂效应下保持变换域中的良好信道稀疏性与稀疏结构。随后，提出了一种基于深度展开的宽带THz大规模MIMO信道估计算法。在酉近似消息传递-稀疏贝叶斯学习算法的每次迭代中，通过深度神经网络（DNN）学习最优更新规则，且该DNN的结构经过定制以有效利用信道内在模式。此外，基于DNN结构与损失函数的新颖设计，开发了一种混合训练方法，以有效训练来自不同系统配置的数据。仿真结果验证了所提算法在性能、复杂度与鲁棒性方面的优越性。