Collaborative perception aims to mitigate the limitations of single-agent perception, such as occlusions, by facilitating data exchange among multiple agents. However, most current works consider a homogeneous scenario where all agents use identity sensors and perception models. In reality, heterogeneous agent types may continually emerge and inevitably face a domain gap when collaborating with existing agents. In this paper, we introduce a new open heterogeneous problem: how to accommodate continually emerging new heterogeneous agent types into collaborative perception, while ensuring high perception performance and low integration cost? To address this problem, we propose HEterogeneous ALliance (HEAL), a novel extensible collaborative perception framework. HEAL first establishes a unified feature space with initial agents via a novel multi-scale foreground-aware Pyramid Fusion network. When heterogeneous new agents emerge with previously unseen modalities or models, we align them to the established unified space with an innovative backward alignment. This step only involves individual training on the new agent type, thus presenting extremely low training costs and high extensibility. To enrich agents' data heterogeneity, we bring OPV2V-H, a new large-scale dataset with more diverse sensor types. Extensive experiments on OPV2V-H and DAIR-V2X datasets show that HEAL surpasses SOTA methods in performance while reducing the training parameters by 91.5% when integrating 3 new agent types. We further implement a comprehensive codebase at: https://github.com/yifanlu0227/HEAL

翻译：协同感知旨在通过多智能体间的数据交换，缓解单一智能体感知存在的遮挡等局限性。然而，当前多数研究考虑的是同构场景，即所有智能体使用相同的传感器和感知模型。现实中，异构智能体类型可能持续涌现，且在与现有智能体协作时不可避免地面临领域差异。本文提出一个新的开放异构问题：如何使持续涌现的新型异构智能体类型融入协同感知，同时确保高感知性能与低集成成本？为解决该问题，我们提出异构联盟（HEAL）——一种新型可扩展协同感知框架。HEAL首先通过创新的多尺度前景感知金字塔融合网络，在初始智能体间建立统一特征空间。当具有未知模态或模型的异构新型智能体出现时，我们采用创新的反向对齐方法将其对齐至已建立的统一空间。该步骤仅需对新型智能体进行单独训练，因此具有极低的训练成本与高扩展性。为丰富智能体的数据异构性，我们构建了包含更多样化传感器类型的大规模新数据集OPV2V-H。在OPV2V-H与DAIR-V2X数据集上的大量实验表明，HEAL在性能上超越当前最优方法，且在集成3种新型智能体时训练参数量减少91.5%。我们已在https://github.com/yifanlu0227/HEAL 提供完整代码库。