While recent Transformer-based approaches have shown impressive performances on event-based object detection tasks, their high computational costs still diminish the low power consumption advantage of event cameras. Image-based works attempt to reduce these costs by introducing sparse Transformers. However, they display inadequate sparsity and adaptability when applied to event-based object detection, since these approaches cannot balance the fine granularity of token-level sparsification and the efficiency of window-based Transformers, leading to reduced performance and efficiency. Furthermore, they lack scene-specific sparsity optimization, resulting in information loss and a lower recall rate. To overcome these limitations, we propose the Scene Adaptive Sparse Transformer (SAST). SAST enables window-token co-sparsification, significantly enhancing fault tolerance and reducing computational overhead. Leveraging the innovative scoring and selection modules, along with the Masked Sparse Window Self-Attention, SAST showcases remarkable scene-aware adaptability: It focuses only on important objects and dynamically optimizes sparsity level according to scene complexity, maintaining a remarkable balance between performance and computational cost. The evaluation results show that SAST outperforms all other dense and sparse networks in both performance and efficiency on two large-scale event-based object detection datasets (1Mpx and Gen1). Code: https://github.com/Peterande/SAST

翻译：尽管近期基于Transformer的方法在事件驱动的目标检测任务中展现了卓越性能，但其高昂的计算成本仍削弱了事件相机低功耗优势。基于图像的方法尝试通过引入稀疏Transformer来降低这些成本，然而当应用于事件目标检测时，这些方法在令牌级稀疏化的细粒度与窗口式Transformer的效率之间难以取得平衡，导致性能与效率双双下降。此外，它们缺乏针对场景的稀疏性优化，造成信息丢失与召回率降低。为克服这些局限，我们提出场景自适应稀疏Transformer（SAST）。SAST实现了窗口-令牌协同稀疏化，显著增强了容错能力并降低了计算开销。借助创新的评分与选择模块以及掩码稀疏窗口自注意力机制，SAST展现出卓越的场景感知自适应能力：它仅关注重要目标，并根据场景复杂度动态优化稀疏程度，在性能与计算成本间维持卓越平衡。评估结果显示，在两个大规模事件目标检测数据集（1Mpx和Gen1）上，SAST在性能与效率两方面均超越所有其他密集与稀疏网络。代码地址：https://github.com/Peterande/SAST