Opportunistic spectrum access has the potential to increase the efficiency of spectrum utilization in cognitive radio networks (CRNs). In CRNs, both spectrum sensing and resource allocation (SSRA) are critical to maximizing system throughput while minimizing collisions of secondary users with the primary network. However, many works in dynamic spectrum access do not consider the impact of imperfect sensing information such as mis-detected channels, which the additional information available in joint SSRA can help remediate. In this work, we examine joint SSRA as an optimization which seeks to maximize a CRN's net communication rate subject to constraints on channel sensing, channel access, and transmit power. Given the non-trivial nature of the problem, we leverage multi-agent reinforcement learning to enable a network of secondary users to dynamically access unoccupied spectrum via only local test statistics, formulated under the energy detection paradigm of spectrum sensing. In doing so, we develop a novel multi-agent implementation of hybrid soft actor critic, MHSAC, based on the QMIX mixing scheme. Through experiments, we find that our SSRA algorithm, HySSRA, is successful in maximizing the CRN's utilization of spectrum resources while also limiting its interference with the primary network, and outperforms the current state-of-the-art by a wide margin. We also explore the impact of wireless variations such as coherence time on the efficacy of the system.

翻译：机会式频谱接入有潜力提升认知无线电网络中频谱利用效率。在认知无线电网络中，频谱感知与资源分配对最大化系统吞吐量及减少次级用户与主网络冲突均至关重要。然而，现有动态频谱接入研究大多未考虑不完美感知信息（如漏检信道）的影响，而联合频谱感知与资源分配中的额外信息可辅助弥补此不足。本研究将联合频谱感知与资源分配视为一个优化问题，旨在满足信道感知、信道接入和发射功率约束条件下最大化认知无线电网络的净通信速率。鉴于该问题的复杂性，我们采用多智能体强化学习，使次级用户网络仅通过本地检验统计量（基于能量检测的频谱感知范式）动态接入空闲频谱。为此，我们提出一种基于QMIX混合机制的新型多智能体混合软演员-评论家算法——MHSAC。实验表明，所提频谱感知与资源分配算法HySSRA能在限制对主网络干扰的同时成功最大化认知无线电网络频谱资源利用率，且性能显著超越当前最先进算法。我们还研究了相干时间等无线信道变化对系统效能的影响。