Collaborative perception improves task performance by expanding the perception range through information sharing among agents. . Immutable heterogeneity poses a significant challenge in collaborative perception, as participating agents may employ different and fixed perception models. This leads to domain gaps in the intermediate features shared among agents, consequently degrading collaborative performance. Aligning the features of all agents to a common representation can eliminate domain gaps with low training cost. However, in existing methods, the common representation is designated as the representation of a specific agent, making it difficult for agents with significant domain discrepancies from this specific agent to achieve proper alignment. This paper proposes NegoCollab, a heterogeneous collaboration method based on the negotiated common representation. It introduces a negotiator during training to derive the common representation from the local representations of each modality's agent, effectively reducing the inherent domain gap with the various local representations. In NegoCollab, the mutual transformation of features between the local representation space and the common representation space is achieved by a pair of sender and receiver. To better align local representations to the common representation containing multimodal information, we introduce structural alignment loss and pragmatic alignment loss in addition to the distribution alignment loss to supervise the training. This enables the knowledge in the common representation to be fully distilled into the sender.

翻译：协同感知通过智能体间的信息共享扩展感知范围，从而提升任务性能。不可变异构性是协同感知面临的重要挑战，因为参与智能体可能采用不同且固定的感知模型。这导致智能体间共享的中间特征存在领域差异，进而降低协同性能。将所有智能体的特征对齐到通用表征可以低成本消除领域差异。然而，现有方法将通用表征指定为特定智能体的表征，导致与该智能体存在显著领域差异的其他智能体难以实现有效对齐。本文提出NegoCollab——一种基于协商式通用表征的异构协同方法。该方法在训练阶段引入协商器，从各模态智能体的局部表征中推导出通用表征，有效缩小了与各类局部表征之间的固有领域差异。在NegoCollab中，特征在局部表征空间与通用表征空间之间的相互转换通过发送器-接收器对实现。为更好地将局部表征对齐至包含多模态信息的通用表征，我们在分布对齐损失的基础上引入结构对齐损失与语用对齐损失来监督训练，从而使通用表征中的知识能够被充分蒸馏至发送器。