Using stickers in online chatting is very prevalent on social media platforms, where the stickers used in the conversation can express someone's intention/emotion/attitude in a vivid, tactful, and intuitive way. Existing sticker retrieval research typically retrieves stickers based on context and the current utterance delivered by the user. That is, the stickers serve as a supplement to the current utterance. However, in the real-world scenario, using stickers to express what we want to say rather than as a supplement to our words only is also important. Therefore, in this paper, we create a new dataset for sticker retrieval in conversation, called StickerInt, where stickers are used to reply to previous conversations or supplement our words. Based on the created dataset, we present a simple yet effective framework for sticker retrieval in conversation based on the learning of intention and the cross-modal relationships between conversation context and stickers, coined as \textbf{Int-RA}. Specifically, we first devise a knowledge-enhanced intention predictor to introduce the intention information into the conversation representations. Subsequently, a relation-aware sticker selector is devised to retrieve the response sticker via cross-modal relationships. Extensive experiments on the created dataset show that the proposed model achieves state-of-the-art performance in sticker retrieval.
翻译:在在线聊天中使用贴纸在社交媒体平台上非常普遍,用户通过贴纸能够以生动、委婉且直观的方式表达意图/情感/态度。现有贴纸检索研究通常基于对话上下文和用户当前话语来检索贴纸,即贴纸仅作为当前话语的补充。然而,在真实场景中,用贴纸表达我们想说的内容而不仅仅是对文字的补充同样重要。因此,本文构建了一个用于对话中贴纸检索的新数据集StickerInt,其中贴纸既可用于回复前文对话,也可作为话语补充。基于该数据集,我们提出了一种简单而有效的对话贴纸检索框架,通过意图学习以及对话上下文与贴纸之间的跨模态关系实现,命名为\textbf{Int-RA}。具体而言,我们首先设计了一个知识增强的意图预测器,将意图信息引入对话表示中;随后,通过跨模态关系设计了一个关系感知的贴纸选择器来检索回复贴纸。在创建的数据集上的大量实验表明,所提模型在贴纸检索任务上达到了当前最优性能。