Cooperative perception can effectively enhance individual perception performance by providing additional viewpoint and expanding the sensing field. Existing cooperation paradigms are either interpretable (result cooperation) or flexible (feature cooperation). In this paper, we propose the concept of query cooperation to enable interpretable instance-level flexible feature interaction. To specifically explain the concept, we propose a cooperative perception framework, termed QUEST, which let query stream flow among agents. The cross-agent queries are interacted via fusion for co-aware instances and complementation for individual unaware instances. Taking camera-based vehicle-infrastructure perception as a typical practical application scene, the experimental results on the real-world dataset, DAIR-V2X-Seq, demonstrate the effectiveness of QUEST and further reveal the advantage of the query cooperation paradigm on transmission flexibility and robustness to packet dropout. We hope our work can further facilitate the cross-agent representation interaction for better cooperative perception in practice.
翻译:协同感知可通过提供额外视角并扩展感知范围来有效提升个体感知性能。现有协作范式或可解释(结果协作)或灵活(特征协作)。本文提出查询协作概念,以实现可解释的实例级灵活特征交互。为具体阐释该概念,我们构建了名为QUEST的协同感知框架,使查询流在智能体间流动。跨智能体的查询通过融合进行共知实例交互,并通过补全处理个体未知实例。以基于摄像头的车路协同感知为典型实际应用场景,在真实数据集DAIR-V2X-Seq上的实验结果验证了QUEST的有效性,并进一步揭示了查询协作范式在传输灵活性和抗丢包鲁棒性方面的优势。希望本工作能促进实际应用中跨智能体表征交互,以实现更优的协同感知。