In this paper, we first assess and harness various Vision Foundation Models (VFMs) in the context of Domain Generalized Semantic Segmentation (DGSS). Driven by the motivation that Leveraging Stronger pre-trained models and Fewer trainable parameters for Superior generalizability, we introduce a robust fine-tuning approach, namely Rein, to parameter-efficiently harness VFMs for DGSS. Built upon a set of trainable tokens, each linked to distinct instances, Rein precisely refines and forwards the feature maps from each layer to the next layer within the backbone. This process produces diverse refinements for different categories within a single image. With fewer trainable parameters, Rein efficiently fine-tunes VFMs for DGSS tasks, surprisingly surpassing full parameter fine-tuning. Extensive experiments across various settings demonstrate that Rein significantly outperforms state-of-the-art methods. Remarkably, with just an extra 1% of trainable parameters within the frozen backbone, Rein achieves a mIoU of 78.4% on the Cityscapes, without accessing any real urban-scene datasets.Code is available at https://github.com/w1oves/Rein.git.

翻译：本文首次评估并利用多种视觉基础模型在域泛化语义分割任务中的表现。基于"利用更强的预训练模型与更少的可训练参数实现更优泛化能力"这一动机，我们提出了一种鲁棒微调方法——Rein，以参数高效方式驾驭视觉基础模型完成域泛化语义分割。该方法基于一组与不同实例关联的可训练令牌，在骨干网络中精准地细化并将每层特征图传递至下一层，从而对单张图像中不同类别产生差异化的特征精炼。通过更少的可训练参数，Rein能够高效微调视觉基础模型应对域泛化语义分割任务，并惊人地超越全参数微调效果。跨多种设置的广泛实验表明，Rein显著优于现有最优方法。值得关注的是，在冻结骨干网络中仅增加1%的可训练参数，Rein在不访问任何真实城市场景数据集的情况下，便在Cityscapes数据集上实现了78.4%的平均交并比。代码开源地址：https://github.com/w1oves/Rein.git。