In numerous artificial intelligence applications, the collaborative efforts of multiple intelligent agents are imperative for the successful attainment of target objectives. To enhance coordination among these agents, a distributed communication framework is often employed. However, indiscriminate information sharing among all agents can be resource-intensive, and the adoption of manually pre-defined communication architectures imposes constraints on inter-agent communication, thus limiting the potential for effective collaboration. Moreover, the communication framework often remains static during inference, which may result in sustained high resource consumption, as in most cases, only key decisions necessitate information sharing among agents. In this study, we introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We formulate this problem as the task of determining the communication graph while enabling the architecture parameters to update normally, thus necessitating a bi-level optimization process. Utilizing continuous relaxation of the graph representation and incorporating attention units, our proposed approach, CommFormer, efficiently optimizes the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner. Additionally, we introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time, based on current observations, thus improving decision-making efficiency. Extensive experiments on a variety of cooperative tasks substantiate the robustness of our model across diverse cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies regardless of changes in the number of agents.

翻译：在众多人工智能应用中，多个智能体的协同努力对于成功实现目标至关重要。为增强这些智能体间的协调性，通常采用分布式通信框架。然而，所有智能体间不加区分的信息共享会消耗大量资源，而采用人工预定义的通信架构则限制了智能体间的通信，从而制约了有效协作的潜力。此外，在推理过程中通信框架往往保持静态，这可能导致持续的高资源消耗，因为在大多数情况下，只有关键决策才需要智能体间的信息共享。在本研究中，我们提出了一种新方法，将智能体间的通信架构概念化为可学习的图。我们将该问题形式化为确定通信图的任务，同时使架构参数能够正常更新，因此需要进行双层优化。通过图表示的连续松弛并结合注意力单元，我们提出的CommFormer方法能够高效优化通信图，并以端到端方式通过梯度下降同步优化架构参数。此外，我们为每个智能体引入了时序门控机制，使其能够基于当前观测动态决定在给定时间是否接收共享信息，从而提升决策效率。在各种协作任务上的大量实验证实了我们的模型在不同协作场景中的鲁棒性，无论智能体数量如何变化，智能体都能制定出更协调、更复杂的策略。