Domain generalization~(DG) aims at solving distribution shift problems in various scenes. Existing approaches are based on Convolution Neural Networks (CNNs) or Vision Transformers (ViTs), which suffer from limited receptive fields or quadratic complexities issues. Mamba, as an emerging state space model (SSM), possesses superior linear complexity and global receptive fields. Despite this, it can hardly be applied to DG to address distribution shifts, due to the hidden state issues and inappropriate scan mechanisms. In this paper, we propose a novel framework for DG, named DGMamba, that excels in strong generalizability toward unseen domains and meanwhile has the advantages of global receptive fields, and efficient linear complexity. Our DGMamba compromises two core components: Hidden State Suppressing~(HSS) and Semantic-aware Patch refining~(SPR). In particular, HSS is introduced to mitigate the influence of hidden states associated with domain-specific features during output prediction. SPR strives to encourage the model to concentrate more on objects rather than context, consisting of two designs: Prior-Free Scanning~(PFS), and Domain Context Interchange~(DCI). Concretely, PFS aims to shuffle the non-semantic patches within images, creating more flexible and effective sequences from images, and DCI is designed to regularize Mamba with the combination of mismatched non-semantic and semantic information by fusing patches among domains. Extensive experiments on four commonly used DG benchmarks demonstrate that the proposed DGMamba achieves remarkably superior results to state-of-the-art models. The code will be made publicly available.
翻译:域泛化旨在解决不同场景下的分布偏移问题。现有方法基于卷积神经网络或视觉Transformer,但分别受限于有限感受野与二次复杂度问题。Mamba作为一种新兴的状态空间模型,具备优越的线性复杂度与全局感受野。然而,由于状态空间模型的隐藏状态问题以及不匹配的扫描机制,其难以直接应用于域泛化以解决分布偏移。本文提出一种名为DGMamba的新型域泛化框架,该框架在对未见过域具备强泛化能力的同时,兼具全局感受野与高效线性复杂度的优势。DGMamba包含两大核心组件:隐藏状态抑制模块和语义感知补丁精炼模块。具体而言,HSS通过抑制与域特定特征关联的隐藏状态对输出预测的影响;SPR则致力于促使模型更多关注目标对象而非背景上下文,包含无先验扫描与域上下文互换两种设计。其中,PFS通过打乱图像内非语义补丁的排列顺序,生成更具灵活性与有效性的图像序列;DCI则通过混合不同域的补丁,利用不匹配的非语义与语义信息组合对Mamba进行正则化。在四个常用域泛化基准上的大量实验表明,所提DGMamba的性能显著优于现有最优模型。相关代码将公开发布。