To improve the performance of multi-agent reinforcement learning under the constraint of wireless resources, we propose a message importance metric and design an importance-aware scheduling policy to effectively exchange messages. The key insight is spending the precious communication resources on important messages. The message importance depends not only on the messages themselves, but also on the needs of agents who receive them. Accordingly, we propose a query-message-based architecture, called QMNet. Agents generate queries and messages with the environment observation. Sharing queries can help calculate message importance. Exchanging messages can help agents cooperate better. Besides, we exploit the message importance to deal with random access collisions in decentralized systems. Furthermore, a message prediction mechanism is proposed to compensate for messages that are not transmitted. Finally, we evaluate the proposed schemes in a traffic junction environment, where only a fraction of agents can send messages due to limited wireless resources. Results show that QMNet can extract valuable information to guarantee the system performance even when only $30\%$ of agents can share messages. By exploiting message prediction, the system can further save $40\%$ of wireless resources. The importance-aware decentralized multi-access mechanism can effectively avoid collisions, achieving almost the same performance as centralized scheduling.

翻译：为在无线资源约束下提升多智能体强化学习性能，本文提出一种消息重要性度量方法，并设计了一种重要性感知调度策略以实现高效消息交换。其核心思想是将宝贵的通信资源用于传输重要消息，而消息的重要性不仅取决于消息本身，还与接收智能体的需求密切相关。基于此，我们提出了一种基于查询-消息的架构QMNet。智能体根据环境观测生成查询与消息。共享查询有助于计算消息重要性，交换消息则能增强智能体间的协作。此外，我们利用消息重要性解决去中心化系统中的随机接入冲突问题，并提出一种消息预测机制来补偿未传输的消息。最终，在仅部分智能体因无线资源受限而能发送消息的交通路口环境中评估所提方案。实验结果表明：即使仅有30%的智能体能够共享消息，QMNet仍能提取有价值信息以保证系统性能；借助消息预测，系统可进一步节省40%的无线资源；所提出的重要性感知去中心化多址接入机制能有效避免冲突，性能几乎与集中式调度相当。