Collaboration by the sharing of semantic information is crucial to enable the enhancement of perception capabilities. However, existing collaborative perception methods tend to focus solely on the spatial features of semantic information, while neglecting the importance of the temporal dimension in collaborator selection and semantic information fusion, which instigates performance degradation. In this article, we propose a novel collaborative perception framework, IoSI-CP, which takes into account the importance of semantic information (IoSI) from both temporal and spatial dimensions. Specifically, we develop an IoSI-based collaborator selection method that effectively identifies advantageous collaborators but excludes those that bring negative benefits. Moreover, we present a semantic information fusion algorithm called HPHA (historical prior hybrid attention), which integrates a multi-scale transformer module and a short-term attention module to capture IoSI from spatial and temporal dimensions, and assigns varying weights for efficient aggregation. Extensive experiments on two open datasets demonstrate that our proposed IoSI-CP significantly improves the perception performance compared to state-of-the-art approaches. The code associated with this research is publicly available at https://github.com/huangqzj/IoSI-CP/.
翻译:通过共享语义信息进行协作对于提升感知能力至关重要。然而,现有协同感知方法往往仅关注语义信息的空间特征,而忽视了时间维度在协作者选择与语义信息融合中的重要性,这导致性能下降。本文提出一种新型协同感知框架IoSI-CP,该框架从时间与空间两个维度考量语义信息的重要性(IoSI)。具体而言,我们开发了一种基于IoSI的协作者选择方法,该方法能有效识别带来增益的协作者,同时排除产生负面收益的协作者。此外,我们提出了一种名为HPHA(历史先验混合注意力)的语义信息融合算法,该算法集成了多尺度Transformer模块和短期注意力模块,从空间与时间维度捕获IoSI,并分配不同权重以实现高效聚合。在两个开放数据集上的大量实验表明,与最先进方法相比,我们提出的IoSI-CP显著提升了感知性能。本研究的相关代码已在https://github.com/huangqzj/IoSI-CP/公开提供。