The unique complementarity of frame-based and event cameras for high frame rate object tracking has recently inspired some research attempts to develop multi-modal fusion approaches. However, these methods directly fuse both modalities and thus ignore the environmental attributes, e.g., motion blur, illumination variance, occlusion, scale variation, etc. Meanwhile, no interaction between search and template features makes distinguishing target objects and backgrounds difficult. As a result, performance degradation is induced especially in challenging conditions. This paper proposes a novel and effective Transformer-based event-guided tracking framework, called eMoE-Tracker, which achieves new SOTA performance under various conditions. Our key idea is to disentangle the environment into several learnable attributes to dynamically learn the attribute-specific features for better interaction and discriminability between the target information and background. To achieve the goal, we first propose an environmental Mix-of-Experts (eMoE) module that is built upon the environmental Attributes Disentanglement to learn attribute-specific features and environmental Attributes Gating to assemble the attribute-specific features by the learnable attribute scores dynamically. The eMoE module is a subtle router that fine-tunes the transformer backbone more efficiently. We then introduce a contrastive relation modeling (CRM) module to improve interaction and discriminability between the target information and background. Extensive experiments on diverse event-based benchmark datasets showcase the superior performance of our eMoE-Tracker compared to the prior arts.

翻译：基于帧的相机与事件相机在高帧率目标跟踪中独特的互补性，近期激发了一些研究尝试开发多模态融合方法。然而，这些方法直接融合两种模态，从而忽略了环境属性，例如运动模糊、光照变化、遮挡、尺度变化等。同时，搜索特征与模板特征之间缺乏交互，使得区分目标物体与背景变得困难。因此，尤其在挑战性条件下会导致性能下降。本文提出了一种新颖且有效的基于Transformer的事件引导跟踪框架，称为eMoE-Tracker，该框架在各种条件下实现了新的SOTA性能。我们的核心思想是将环境解耦为若干可学习的属性，以动态学习属性特定的特征，从而改善目标信息与背景之间的交互和可区分性。为实现这一目标，我们首先提出了一个环境混合专家（eMoE）模块，该模块基于环境属性解耦来学习属性特定的特征，并通过环境属性门控动态地利用可学习的属性分数组装属性特定的特征。eMoE模块是一个精巧的路由器，能够更高效地微调Transformer骨干网络。随后，我们引入了对比关系建模（CRM）模块，以增强目标信息与背景之间的交互和可区分性。在多样化基于事件的基准数据集上进行的大量实验表明，我们的eMoE-Tracker相较于现有技术具有卓越的性能。