An end-to-end autoencoder (AE) framework is developed for downlink non-orthogonal multiple access (NOMA) over Rayleigh fading channels, which learns interference-aware and channel-adaptive super-constellations. While existing works either assume additive white Gaussian noise channels or treat fading channels without a fully end-to-end learning approach, our framework directly embeds the wireless channel into both training and inference. To account for practical channel state information (CSI), we further incorporate limited feedback via both uniform and Lloyd-Max quantization of channel gains and analyze their impact on AE training and bit error rate (BER) performance. Simulation results show that, with perfect CSI, the proposed AE outperforms the existing analytical NOMA schemes. In addition, Lloyd-Max quantization achieves superior BER performance compared to uniform quantization. These results demonstrate that end-to-end AEs trained directly over Rayleigh fading can effectively learn robust, interference-aware signaling strategies, paving the way for NOMA deployment in fading environments with realistic CSI constraints.

翻译：本文针对瑞利衰落信道下的下行非正交多址接入（NOMA）系统，开发了一种端到端自编码器框架。该框架能够学习具有干扰感知和信道自适应特性的超星座图。现有研究通常假设加性高斯白噪声信道，或在处理衰落信道时未采用完全端到端的学习方法，而本框架直接将无线信道嵌入训练和推理过程。为考虑实际信道状态信息（CSI）条件，我们进一步通过均匀量化和Lloyd-Max量化对信道增益进行有限反馈建模，并分析其对自编码器训练和误码率性能的影响。仿真结果表明：在完美CSI条件下，所提出的自编码器性能优于现有解析型NOMA方案；同时，与均匀量化相比，Lloyd-Max量化能获得更优的误码率性能。这些结果证明，直接在瑞利衰落信道上训练的端到端自编码器能够有效学习鲁棒且具备干扰感知能力的信号传输策略，为在具有实际CSI约束的衰落环境中部署NOMA技术开辟了新途径。