Cooperative perception can effectively enhance individual perception performance by providing additional viewpoint and expanding the sensing field. Existing cooperation paradigms are either interpretable (result cooperation) or flexible (feature cooperation). In this paper, we propose the concept of query cooperation to enable interpretable instance-level flexible feature interaction. To specifically explain the concept, we propose a cooperative perception framework, termed QUEST, which let query stream flow among agents. The cross-agent queries are interacted via fusion for co-aware instances and complementation for individual unaware instances. Taking camera-based vehicle-infrastructure perception as a typical practical application scene, the experimental results on the real-world dataset, DAIR-V2X-Seq, demonstrate the effectiveness of QUEST and further reveal the advantage of the query cooperation paradigm on transmission flexibility and robustness to packet dropout. We hope our work can further facilitate the cross-agent representation interaction for better cooperative perception in practice.
翻译:协同感知通过提供额外视角并扩展感知范围,可有效提升个体感知性能。现有协作范式要么具备可解释性(结果协作),要么具备灵活性(特征协作)。本文提出查询协作概念,以实现可解释的实例级灵活特征交互。为具体阐释该概念,我们提出名为QUEST的协同感知框架,该框架使查询流在智能体间流动。跨智能体查询通过融合(针对共知实例)与补全(针对个体未知实例)两种方式实现交互。以基于摄像头的车路协同感知作为典型实际应用场景,在真实数据集DAIR-V2X-Seq上的实验结果验证了QUEST的有效性,并进一步揭示了查询协作范式在传输灵活性与抗丢包鲁棒性方面的优势。我们期望该工作能促进跨智能体表征交互,从而在实际场景中实现更优的协同感知。