Image-text retrieval which associates different modalities has drawn broad attention due to its excellent research value and broad real-world application. However, most of the existing methods haven't taken the high-level semantic relationships ("style embedding") and common knowledge from multi-modalities into full consideration. To this end, we introduce a novel style transformer network with common knowledge optimization (CKSTN) for image-text retrieval. The main module is the common knowledge adaptor (CKA) with both the style embedding extractor (SEE) and the common knowledge optimization (CKO) modules. Specifically, the SEE uses the sequential update strategy to effectively connect the features of different stages in SEE. The CKO module is introduced to dynamically capture the latent concepts of common knowledge from different modalities. Besides, to get generalized temporal common knowledge, we propose a sequential update strategy to effectively integrate the features of different layers in SEE with previous common feature units. CKSTN demonstrates the superiorities of the state-of-the-art methods in image-text retrieval on MSCOCO and Flickr30K datasets. Moreover, CKSTN is constructed based on the lightweight transformer which is more convenient and practical for the application of real scenes, due to the better performance and lower parameters.

翻译：图像-文本检索通过关联不同模态信息，因其卓越的研究价值和广泛的现实应用而备受关注。然而，现有方法大多未能充分考虑跨模态的高级语义关系（“风格嵌入”）和常识知识。为此，我们提出一种新颖的基于常识知识优化的风格Transformer网络（CKSTN）用于图像-文本检索。其核心模块是融合风格嵌入提取器（SEE）和常识知识优化（CKO）模块的常识知识适配器（CKA）。具体而言，SEE采用序列更新策略有效连接其不同阶段特征；CKO模块被引入以动态捕获来自不同模态的潜在常识概念。此外，为获取泛化性时序常识知识，我们提出序列更新策略，将SEE中不同层特征与历史常识特征单元有效融合。在MSCOCO和Flickr30K数据集上的实验表明，CKSTN在图像-文本检索任务中展现出超越现有最优方法的优越性。同时，CKSTN基于轻量级Transformer构建，因更优性能与更低参数量，更便于实际场景应用。