Carrier aggregation (CA) is a technique that allows mobile networks to combine multiple carriers to increase user data rate. On the uplink, for power constrained users, this translates to the need for an efficient resource allocation scheme, where each user distributes its available power among its assigned uplink carriers. Choosing a good set of carriers and allocating appropriate power on the carriers is important. If the carrier allocation on the uplink is such that a harmonic of a user's uplink carrier falls on the downlink frequency of that user, it leads to a self coupling-induced sensitivity degradation of that user's downlink receiver. In this paper, we model the uplink carrier aggregation problem as an optimal resource allocation problem with the associated constraints of non-linearities induced self interference (SI). This involves optimization over a discrete variable (which carriers need to be turned on) and a continuous variable (what power needs to be allocated on the selected carriers) in dynamic environments, a problem which is hard to solve using traditional methods owing to the mixed nature of the optimization variables and the additional need to consider the SI constraint. We adopt a reinforcement learning (RL) framework involving a compound-action actor-critic (CA2C) algorithm for the uplink carrier aggregation problem. We propose a novel reward function that is critical for enabling the proposed CA2C algorithm to efficiently handle SI. The CA2C algorithm along with the proposed reward function learns to assign and activate suitable carriers in an online fashion. Numerical results demonstrate that the proposed RL based scheme is able to achieve higher sum throughputs compared to naive schemes. The results also demonstrate that the proposed reward function allows the CA2C algorithm to adapt the optimization both in the presence and absence of SI.

翻译：载波聚合（CA）是一种允许移动网络组合多个载波以提高用户数据速率的技术。在上行链路中，对于功率受限的用户，这转化为对高效资源分配方案的需求，即每个用户将其可用功率分配到其分配的上行链路载波上。选择合适的载波集合并在载波上分配适当的功率至关重要。如果上行链路的载波分配使得用户上行链路载波的谐波落在该用户的下行链路频率上，将导致该用户下行链路接收器因自耦合引起的灵敏度下降。在本文中，我们将上行链路载波聚合问题建模为一个具有非线性引起的自干扰（SI）相关约束的最优资源分配问题。这涉及在动态环境中对离散变量（需要开启哪些载波）和连续变量（在选定载波上分配多少功率）进行优化，由于优化变量的混合性质以及额外需要考虑SI约束，该问题难以使用传统方法解决。我们采用了一种强化学习（RL）框架，包含针对上行链路载波聚合问题的复合动作演员-评论家（CA2C）算法。我们提出了一种新颖的奖励函数，这对于使所提出的CA2C算法能够有效处理SI至关重要。CA2C算法与所提出的奖励函数共同学习以在线方式分配和激活合适的载波。数值结果表明，与简单方案相比，所提出的基于RL的方案能够实现更高的总吞吐量。结果还表明，所提出的奖励函数使CA2C算法能够在存在和不存在SI的情况下自适应地进行优化。