Multi-functional reconfigurable intelligent surface (MF-RIS) is conceived to address the communication efficiency thanks to its extended signal coverage from its active RIS capability and self-sustainability from energy harvesting (EH). We investigate the architecture of multi-MF-RISs to assist non-orthogonal multiple access (NOMA) downlink networks. We formulate an energy efficiency (EE) maximization problem by optimizing power allocation, transmit beamforming and MF-RIS configurations of amplitudes, phase-shifts and EH ratios, as well as the position of MF-RISs, while satisfying constraints of available power, user rate requirements, and self-sustainability property. We design a parametrized sharing scheme for multi-agent hybrid deep reinforcement learning (PMHRL), where the multi-agent proximal policy optimization (PPO) and deep-Q network (DQN) handle continuous and discrete variables, respectively. The simulation results have demonstrated that proposed PMHRL has the highest EE compared to other benchmarks, including cases without parametrized sharing, pure PPO and DQN. Moreover, the proposed multi-MF-RISs-aided downlink NOMA achieves the highest EE compared to scenarios of no-EH/amplification, traditional RISs, and deployment without RISs/MF-RISs under different multiple access.

翻译：多功能可重构智能表面凭借其主动RIS能力带来的扩展信号覆盖范围以及能量收集功能实现的自持特性，被构想用于提升通信效率。本文研究了多MF-RIS辅助非正交多址下行网络的架构。通过联合优化功率分配、发射波束成形、MF-RIS的幅度/相移/能量收集比配置以及MF-RIS的部署位置，在满足可用功率、用户速率需求和自持特性约束的条件下，构建了能效最大化问题。我们设计了一种参数化共享的多智能体混合深度强化学习方案，其中多智能体近端策略优化算法与深度Q网络分别处理连续变量和离散变量。仿真结果表明，与无参数化共享方案、纯PPO及纯DQN等基准方法相比，所提出的PMHRL方案能实现最高的系统能效。此外，在不同多址接入方式下，所提出的多MF-RIS辅助下行NOMA方案相比无能量收集/放大功能的系统、传统RIS方案以及无RIS/MF-RIS部署场景均展现出最优的能效性能。