Collaborative perception allows each agent to enhance its perceptual abilities by exchanging messages with others. It inherently results in a trade-off between perception ability and communication costs. Previous works transmit complete full-frame high-dimensional feature maps among agents, resulting in substantial communication costs. To promote communication efficiency, we propose only transmitting the information needed for the collaborator's downstream task. This pragmatic communication strategy focuses on three key aspects: i) pragmatic message selection, which selects task-critical parts from the complete data, resulting in spatially and temporally sparse feature vectors; ii) pragmatic message representation, which achieves pragmatic approximation of high-dimensional feature vectors with a task-adaptive dictionary, enabling communicating with integer indices; iii) pragmatic collaborator selection, which identifies beneficial collaborators, pruning unnecessary communication links. Following this strategy, we first formulate a mathematical optimization framework for the perception-communication trade-off and then propose PragComm, a multi-agent collaborative perception system with two key components: i) single-agent detection and tracking and ii) pragmatic collaboration. The proposed PragComm promotes pragmatic communication and adapts to a wide range of communication conditions. We evaluate PragComm for both collaborative 3D object detection and tracking tasks in both real-world, V2V4Real, and simulation datasets, OPV2V and V2X-SIM2.0. PragComm consistently outperforms previous methods with more than 32.7K times lower communication volume on OPV2V. Code is available at github.com/PhyllisH/PragComm.

翻译：协同感知允许每个智能体通过与其他智能体交换信息来增强其感知能力，这本质上导致了感知能力与通信成本之间的权衡。以往的工作在智能体间传输完整的全帧高维特征图，导致通信成本高昂。为提升通信效率，我们提出仅传输协作方下游任务所需的信息。这种实用通信策略聚焦三个关键方面：i) 实用消息选择，从完整数据中选取任务关键部分，生成时空稀疏特征向量；ii) 实用消息表示，通过任务自适应字典实现高维特征向量的实用近似，支持用整数索引进行通信；iii) 实用协作者选择，识别有益协作者，修剪不必要的通信链路。基于该策略，我们首先构建了感知-通信权衡的数学优化框架，随后提出PragComm——一种包含两大核心组件的多智能体协同感知系统：i) 单智能体检测与跟踪；ii) 实用协作。所提出的PragComm促进了实用通信，并能适应广泛的通信条件。我们在真实数据集V2V4Real和仿真数据集OPV2V、V2X-SIM2.0上评估了PragComm在协同3D目标检测与跟踪任务中的表现。PragComm在OPV2V上以超过32.7K倍的通信量优势持续优于以往方法。代码已开源至github.com/PhyllisH/PragComm。