Collaborative perception improves task performance by expanding the perception range through information sharing among agents. . Immutable heterogeneity poses a significant challenge in collaborative perception, as participating agents may employ different and fixed perception models. This leads to domain gaps in the intermediate features shared among agents, consequently degrading collaborative performance. Aligning the features of all agents to a common representation can eliminate domain gaps with low training cost. However, in existing methods, the common representation is designated as the representation of a specific agent, making it difficult for agents with significant domain discrepancies from this specific agent to achieve proper alignment. This paper proposes NegoCollab, a heterogeneous collaboration method based on the negotiated common representation. It introduces a negotiator during training to derive the common representation from the local representations of each modality's agent, effectively reducing the inherent domain gap with the various local representations. In NegoCollab, the mutual transformation of features between the local representation space and the common representation space is achieved by a pair of sender and receiver. To better align local representations to the common representation containing multimodal information, we introduce structural alignment loss and pragmatic alignment loss in addition to the distribution alignment loss to supervise the training. This enables the knowledge in the common representation to be fully distilled into the sender.

翻译：协同感知通过智能体间的信息共享扩展感知范围，从而提升任务性能。不可变异构性是协同感知面临的重要挑战，因为参与智能体可能采用不同且固定的感知模型。这导致智能体间共享的中间特征存在域差异，进而降低协同性能。将所有智能体的特征对齐到公共表征可以较低训练成本消除域差异。然而，现有方法将公共表征指定为特定智能体的表征，导致与该智能体存在显著域差异的其他智能体难以实现有效对齐。本文提出NegoCollab，一种基于协商公共表征的异构协同方法。该方法在训练阶段引入协商器，从各模态智能体的局部表征推导出公共表征，有效缩小其与各类局部表征之间的固有域差异。在NegoCollab中，特征在局部表征空间与公共表征空间之间的相互转换通过一对发送器与接收器实现。为更好地将局部表征对齐至包含多模态信息的公共表征，我们在分布对齐损失之外引入结构对齐损失与语用对齐损失来监督训练，从而使公共表征中的知识能够充分蒸馏至发送器中。