Collaborative perception aims to mitigate the limitations of single-agent perception, such as occlusions, by facilitating data exchange among multiple agents. However, most current works consider a homogeneous scenario where all agents use identity sensors and perception models. In reality, heterogeneous agent types may continually emerge and inevitably face a domain gap when collaborating with existing agents. In this paper, we introduce a new open heterogeneous problem: how to accommodate continually emerging new heterogeneous agent types into collaborative perception, while ensuring high perception performance and low integration cost? To address this problem, we propose HEterogeneous ALliance (HEAL), a novel extensible collaborative perception framework. HEAL first establishes a unified feature space with initial agents via a novel multi-scale foreground-aware Pyramid Fusion network. When heterogeneous new agents emerge with previously unseen modalities or models, we align them to the established unified space with an innovative backward alignment. This step only involves individual training on the new agent type, thus presenting extremely low training costs and high extensibility. It also protects new agents' model details from disclosure since the training can be conducted by the agent owner locally. To enrich agents' data heterogeneity, we bring OPV2V-H, a new large-scale dataset with more diverse sensor types. Extensive experiments on OPV2V-H and DAIR-V2X datasets show that HEAL surpasses SOTA methods in performance while reducing the training parameters by 91.5% when integrating 3 new agent types. Code and data are available at: https://github.com/yifanlu0227/HEAL.

翻译：协同感知旨在通过促进多智能体间的数据交换来缓解单一智能体感知的局限性（例如遮挡问题）。然而，当前大多数研究考虑的是同构场景，即所有智能体使用相同的传感器和感知模型。现实中，异构智能体类型可能持续涌现，并在与现有智能体协作时不可避免地面临领域差异。本文提出一个新的开放异构问题：如何在确保高感知性能与低集成成本的前提下，容纳持续涌现的新型异构智能体类型参与协同感知？针对该问题，我们提出异构联盟（HEAL）——一种新颖的可扩展协同感知框架。HEAL首先通过创新的多尺度前景感知金字塔融合网络，利用初始智能体建立统一特征空间。当具有前所未见模态或模型的新型异构智能体出现时，我们通过创新的反向对齐方法将其对齐至已建立的统一空间。该步骤仅涉及对新智能体类型的独立训练，因此具有极低的训练成本与高扩展性，同时由于训练可由智能体所有者本地执行，还可避免新智能体模型细节的泄露。为丰富智能体数据异构性，我们提出OPV2V-H——一个包含更多样化传感器类型的大规模新数据集。在OPV2V-H与DAIR-V2X数据集上的大量实验表明，HEAL在性能上超越当前最先进方法，同时在集成3种新型智能体类型时减少91.5%的训练参数。代码与数据见：https://github.com/yifanlu0227/HEAL。