Fashion vision-language pre-training models have shown efficacy for a wide range of downstream tasks. However, general vision-language pre-training models pay less attention to fine-grained domain features, while these features are important in distinguishing the specific domain tasks from general tasks. We propose a method for fine-grained fashion vision-language pre-training based on fashion Symbols and Attributes Prompt (FashionSAP) to model fine-grained multi-modalities fashion attributes and characteristics. Firstly, we propose the fashion symbols, a novel abstract fashion concept layer, to represent different fashion items and to generalize various kinds of fine-grained fashion features, making modelling fine-grained attributes more effective. Secondly, the attributes prompt method is proposed to make the model learn specific attributes of fashion items explicitly. We design proper prompt templates according to the format of fashion data. Comprehensive experiments are conducted on two public fashion benchmarks, i.e., FashionGen and FashionIQ, and FashionSAP gets SOTA performances for four popular fashion tasks. The ablation study also shows the proposed abstract fashion symbols, and the attribute prompt method enables the model to acquire fine-grained semantics in the fashion domain effectively. The obvious performance gains from FashionSAP provide a new baseline for future fashion task research.

翻译：时尚视觉语言预训练模型已在多种下游任务中展现出有效性。然而，通用视觉语言预训练模型对细粒度领域特征的关注不足，而这些特征正是区分特定领域任务与通用任务的关键。本文提出一种基于时尚符号与属性提示（FashionSAP）的细粒度时尚视觉语言预训练方法，用于建模细粒度多模态时尚属性与特征。首先，我们提出时尚符号这一新型抽象时尚概念层，用以表征不同时尚单品并泛化各类细粒度时尚特征，使细粒度属性建模更为高效。其次，提出属性提示方法，使模型能够显式学习时尚单品的特定属性。我们根据时尚数据格式设计了恰当的提示模板。在FashionGen与FashionIQ两个公开时尚基准数据集上进行了全面实验，FashionSAP在四个主流时尚任务中均取得最优性能。消融研究进一步表明，所提出的抽象时尚符号与属性提示方法能有效引导模型习得时尚领域细粒度语义。FashionSAP带来的显著性能提升为未来时尚任务研究提供了新基准。