Collaborative perception improves task performance by expanding the perception range through information sharing among agents. . Immutable heterogeneity poses a significant challenge in collaborative perception, as participating agents may employ different and fixed perception models. This leads to domain gaps in the intermediate features shared among agents, consequently degrading collaborative performance. Aligning the features of all agents to a common representation can eliminate domain gaps with low training cost. However, in existing methods, the common representation is designated as the representation of a specific agent, making it difficult for agents with significant domain discrepancies from this specific agent to achieve proper alignment. This paper proposes NegoCollab, a heterogeneous collaboration method based on the negotiated common representation. It introduces a negotiator during training to derive the common representation from the local representations of each modality's agent, effectively reducing the inherent domain gap with the various local representations. In NegoCollab, the mutual transformation of features between the local representation space and the common representation space is achieved by a pair of sender and receiver. To better align local representations to the common representation containing multimodal information, we introduce structural alignment loss and pragmatic alignment loss in addition to the distribution alignment loss to supervise the training. This enables the knowledge in the common representation to be fully distilled into the sender.

翻译：协同感知通过智能体间的信息共享扩展感知范围，从而提升任务性能。然而，不可变的异构性为协同感知带来了显著挑战，因为参与智能体可能采用不同且固定的感知模型。这导致智能体间共享的中间特征存在领域差异，进而降低协同性能。将所有智能体的特征对齐到一个共同表征可以以较低训练成本消除领域差异。然而，现有方法通常将共同表征指定为某个特定智能体的表征，这使得与该智能体领域差异较大的其他智能体难以实现有效对齐。本文提出NegoCollab，一种基于协商共同表征的异构协同方法。该方法在训练阶段引入一个协商器，从各模态智能体的局部表征中推导出共同表征，有效减小了与不同局部表征之间的固有领域差异。在NegoCollab中，特征在局部表征空间与共同表征空间之间的相互转换通过一对发送器和接收器实现。为了更好地将局部表征对齐到包含多模态信息的共同表征，我们在分布对齐损失的基础上引入了结构对齐损失和语用对齐损失来监督训练，从而使共同表征中的知识能够充分蒸馏到发送器中。