This study explores the application of the rate-splitting multiple access (RSMA) technique, vital for interference mitigation in modern communication systems. It investigates the use of precoding methods in RSMA, especially in complex multiple-antenna interference channels, employing deep reinforcement learning. The aim is to optimize precoders and power allocation for common and private data streams involving multiple decision-makers. A multi-agent deep deterministic policy gradient (MADDPG) framework is employed to address this complexity, where decentralized agents collectively learn to optimize actions in a continuous policy space. We also explore the challenges posed by imperfect channel side information at the transmitter. Additionally, decoding order estimation is addressed to determine the optimal decoding sequence for common and private data sequences. Simulation results demonstrate the effectiveness of the proposed RSMA method based on MADDPG, achieving the upper bound in single-antenna scenarios and closely approaching theoretical limits in multi-antenna scenarios. Comparative analysis shows superiority over other techniques such as MADDPG without rate-splitting, maximal ratio transmission (MRT), zero-forcing (ZF), and leakage-based precoding methods. These findings highlight the potential of deep reinforcement learning-driven RSMA in reducing interference and enhancing system performance in communication systems.

翻译：本研究探讨了速率分割多址接入技术在干扰抑制中的应用，该技术对现代通信系统至关重要。研究重点考察了在复杂多天线干扰信道中，结合深度强化学习使用预编码方法的RSMA技术。其目标是为涉及多个决策者的公共与私有数据流优化预编码器与功率分配。采用多智能体深度确定性策略梯度框架以应对这一复杂性，其中分散的智能体通过集体学习在连续策略空间中优化行动。我们还探讨了发射机侧信道状态信息不完善所带来的挑战。此外，针对公共与私有数据序列的最优解码顺序确定问题，研究提出了解码顺序估计方法。仿真结果表明，所提出的基于MADDPG的RSMA方法具有显著有效性，在单天线场景中达到了理论上限，在多天线场景中则紧密逼近理论极限。对比分析显示，该方法优于其他技术，如无速率分割的MADDPG、最大比传输、迫零以及基于泄漏的预编码方法。这些发现凸显了深度强化学习驱动的RSMA在降低通信系统干扰与提升系统性能方面的潜力。