Referring image segmentation aims to segment the image region of interest according to the given language expression, which is a typical multi-modal task. Existing methods either adopt the pixel classification-based or the learnable query-based framework for mask generation, both of which are insufficient to deal with various text-image pairs with a fix number of parametric prototypes. In this work, we propose an end-to-end framework built on transformer to perform Linguistic query-Guided mask generation, dubbed LGFormer. It views the linguistic features as query to generate a specialized prototype for arbitrary input image-text pair, thus generating more consistent segmentation results. Moreover, we design several cross-modal interaction modules (\eg, vision-language bidirectional attention module, VLBA) in both encoder and decoder to achieve better cross-modal alignment.

翻译：指代图像分割旨在根据给定的语言表达分割出图像中的感兴趣区域，这是一项典型的多模态任务。现有方法要么采用基于像素分类的框架，要么采用基于可学习查询的框架来生成掩码，这两种方法都难以用固定数量的参数原型来处理各种文本-图像对。在这项工作中，我们提出了一种基于Transformer的端到端框架，用于执行语言查询引导的掩码生成，命名为LGFormer。它将语言特征视为查询，为任意输入图像-文本对生成专门的原型，从而产生更一致的分割结果。此外，我们在编码器和解码器中设计了多个跨模态交互模块（例如，视觉-语言双向注意力模块VLBA），以实现更好的跨模态对齐。