In a joint vision-language space, a text feature (e.g., from "a photo of a dog") could effectively represent its relevant image features (e.g., from dog photos). Also, a recent study has demonstrated the cross-modal transferability phenomenon of this joint space. From these observations, we propose PromptStyler which simulates various distribution shifts in the joint space by synthesizing diverse styles via prompts without using any images to deal with source-free domain generalization. The proposed method learns to generate a variety of style features (from "a S* style of a") via learnable style word vectors for pseudo-words S*. To ensure that learned styles do not distort content information, we force style-content features (from "a S* style of a [class]") to be located nearby their corresponding content features (from "[class]") in the joint vision-language space. After learning style word vectors, we train a linear classifier using synthesized style-content features. PromptStyler achieves the state of the art on PACS, VLCS, OfficeHome and DomainNet, even though it does not require any images for training.
翻译:在联合视觉-语言空间中,文本特征(例如来自"一张狗的照片")能够有效表征其相关的图像特征(例如来自狗照片)。此外,近期研究证明了该联合空间的跨模态可迁移性现象。基于这些观察,我们提出PromptStyler,通过提示合成多样风格来模拟联合空间中的各种分布偏移,无需使用任何图像即可处理无源域泛化问题。该方法通过可学习的风格词向量学习生成多样风格特征(来自"一个S*风格的"),其中S*为伪词。为确保所学风格不扭曲内容信息,我们强制风格-内容特征(来自"一个S*风格的[类别]")在联合视觉-语言空间中与其对应的内容特征(来自"[类别]")保持邻近。在学习风格词向量后,我们使用合成的风格-内容特征训练线性分类器。PromptStyler在PACS、VLCS、OfficeHome和DomainNet上达到了最先进性能,尽管其训练无需任何图像。