Opportunistic spectrum access has the potential to increase the efficiency of spectrum utilization in cognitive radio networks (CRNs). In CRNs, both spectrum sensing and resource allocation (SSRA) are critical to maximizing system throughput while minimizing collisions of secondary users with the primary network. However, many works in dynamic spectrum access do not consider the impact of imperfect sensing information such as mis-detected channels, which the additional information available in joint SSRA can help remediate. In this work, we examine joint SSRA as an optimization which seeks to maximize a CRN's net communication rate subject to constraints on channel sensing, channel access, and transmit power. Given the non-trivial nature of the problem, we leverage multi-agent reinforcement learning to enable a network of secondary users to dynamically access unoccupied spectrum via only local test statistics, formulated under the energy detection paradigm of spectrum sensing. In doing so, we develop a novel multi-agent implementation of hybrid soft actor critic, MHSAC, based on the QMIX mixing scheme. Through experiments, we find that our SSRA algorithm, HySSRA, is successful in maximizing the CRN's utilization of spectrum resources while also limiting its interference with the primary network, and outperforms the current state-of-the-art by a wide margin. We also explore the impact of wireless variations such as coherence time on the efficacy of the system.

翻译：机会频谱接入有望提高认知无线电网络（CRN）的频谱利用效率。在认知无线电网络中，频谱感知与资源分配（SSRA）对于最大化系统吞吐量同时最小化次级用户与主网络之间的碰撞至关重要。然而，动态频谱接入领域的许多研究未考虑不完美感知信息（如信道漏检）的影响，而联合SSRA中可获得的额外信息有助于缓解此问题。本研究将联合SSRA建模为一种优化问题，其目标是在信道感知、信道接入和发射功率的约束下最大化认知无线电网络的净通信速率。鉴于该问题的非平凡特性，我们采用多智能体强化学习，使次级用户网络能够仅通过局部测试统计量（基于频谱感知的能量检测范式构建）动态接入空闲频谱。为此，我们基于QMIX混合方案开发了一种新颖的混合软演员评论家多智能体实现方法——MHSAC。实验表明，我们的SSRA算法HySSRA能成功最大化认知无线电网络对频谱资源的利用，同时限制对主网络的干扰，并以显著优势超越当前最优方法。我们还探讨了相干时间等无线信道变化对系统效能的影响。