Domain generalization~(DG) aims at solving distribution shift problems in various scenes. Existing approaches are based on Convolution Neural Networks (CNNs) or Vision Transformers (ViTs), which suffer from limited receptive fields or quadratic complexities issues. Mamba, as an emerging state space model (SSM), possesses superior linear complexity and global receptive fields. Despite this, it can hardly be applied to DG to address distribution shifts, due to the hidden state issues and inappropriate scan mechanisms. In this paper, we propose a novel framework for DG, named DGMamba, that excels in strong generalizability toward unseen domains and meanwhile has the advantages of global receptive fields, and efficient linear complexity. Our DGMamba compromises two core components: Hidden State Suppressing~(HSS) and Semantic-aware Patch refining~(SPR). In particular, HSS is introduced to mitigate the influence of hidden states associated with domain-specific features during output prediction. SPR strives to encourage the model to concentrate more on objects rather than context, consisting of two designs: Prior-Free Scanning~(PFS), and Domain Context Interchange~(DCI). Concretely, PFS aims to shuffle the non-semantic patches within images, creating more flexible and effective sequences from images, and DCI is designed to regularize Mamba with the combination of mismatched non-semantic and semantic information by fusing patches among domains. Extensive experiments on four commonly used DG benchmarks demonstrate that the proposed DGMamba achieves remarkably superior results to state-of-the-art models. The code will be made publicly available.
翻译:领域泛化(DG)旨在解决不同场景下的分布偏移问题。现有方法基于卷积神经网络(CNN)或视觉Transformer(ViT),但前者受限于有限感受野,后者存在二次复杂度问题。新兴的状态空间模型(SSM)Mamba具有优越的线性复杂度和全局感受野,然而受隐藏状态问题和不恰当扫描机制的限制,难以直接应用于DG处理分布偏移。本文提出名为DGMamba的新型DG框架,该框架不仅具有强大的面向未见领域的泛化能力,同时兼具全局感受野与高效线性复杂度的优势。DGMamba包含两个核心组件:隐藏状态抑制(HSS)与语义感知补丁细化(SPR)。其中,HSS旨在抑制与领域特定特征相关的隐藏状态对输出预测的影响;SPR由无先验扫描(PFS)和领域上下文互换(DCI)两个设计组成,致力于引导模型更关注对象而非上下文信息。具体而言,PFS通过打乱图像中非语义补丁的排序生成更灵活有效的序列,DCI则通过融合跨领域补丁,利用不匹配的非语义与语义信息对Mamba进行正则化。在四个常用DG基准上的大量实验表明,本文提出的DGMamba取得了显著优于现有最优模型的结果。相关代码将公开发布。