In a joint vision-language space, a text feature (e.g., from "a photo of a dog") could effectively represent its relevant image features (e.g., from dog photos). Inspired by this, we propose PromptStyler which simulates various distribution shifts in the joint space by synthesizing diverse styles via prompts without using any images to deal with source-free domain generalization. Our method learns to generate a variety of style features (from "a S* style of a") via learnable style word vectors for pseudo-words S*. To ensure that learned styles do not distort content information, we force style-content features (from "a S* style of a [class]") to be located nearby their corresponding content features (from "[class]") in the joint vision-language space. After learning style word vectors, we train a linear classifier using synthesized style-content features. PromptStyler achieves the state of the art on PACS, VLCS, OfficeHome and DomainNet, although it does not require any images and takes just ~30 minutes for training using a single GPU.
翻译:在联合视觉-语言空间中,文本特征(例如“一只狗的照片”所对应的特征)能够有效地表征其相关的图像特征(例如狗的照片特征)。受此启发,我们提出PromptStyler,该方法通过提示词合成多样化的风格特征来模拟联合空间中的各种分布偏移,且无需使用任何图像,从而解决无源域泛化问题。我们的方法通过可学习的风格词向量(用于伪词S*)学习生成多种风格特征(来自“a S* style of a”)。为确保学习到的风格不会扭曲内容信息,我们强制风格-内容特征(来自“a S* style of a [class]”)在联合视觉-语言空间中尽可能靠近其对应的内容特征(来自“[class]”)。在学习了风格词向量之后,我们使用合成的风格-内容特征训练线性分类器。尽管PromptStyler无需任何图像,且仅需使用单个GPU即可在大约30分钟内完成训练,但它仍在PACS、VLCS、OfficeHome和DomainNet数据集上达到了当前最优性能。