Memes are a powerful tool for communication over social media. Their affinity for evolving across politics, history, and sociocultural phenomena makes them an ideal communication vehicle. To comprehend the subtle message conveyed within a meme, one must understand the background that facilitates its holistic assimilation. Besides digital archiving of memes and their metadata by a few websites like knowyourmeme.com, currently, there is no efficient way to deduce a meme's context dynamically. In this work, we propose a novel task, MEMEX - given a meme and a related document, the aim is to mine the context that succinctly explains the background of the meme. At first, we develop MCC (Meme Context Corpus), a novel dataset for MEMEX. Further, to benchmark MCC, we propose MIME (MultImodal Meme Explainer), a multimodal neural framework that uses common sense enriched meme representation and a layered approach to capture the cross-modal semantic dependencies between the meme and the context. MIME surpasses several unimodal and multimodal systems and yields an absolute improvement of ~ 4% F1-score over the best baseline. Lastly, we conduct detailed analyses of MIME's performance, highlighting the aspects that could lead to optimal modeling of cross-modal contextual associations.

翻译：模因是社交媒体上强大的沟通工具。它们在政治、历史和社会文化现象中的演变特性使其成为理想的传播载体。要理解模因传递的微妙信息，必须掌握促进其整体理解所需背景知识。目前，除了knowyourmeme.com等少数网站对模因及其元数据进行数字归档外，尚无高效方法动态推断模因上下文。本文提出新任务MEMEX——给定模因及相关文档，目标是挖掘简洁解释模因背景的上下文。首先，我们构建了MEMEX专用数据集MCC（Meme Context Corpus）。为进一步对MCC进行基准测试，我们提出多模态神经框架MIME（MultImodal Meme Explainer），该框架利用常识增强的模因表示和分层方法捕捉模因与上下文间的跨模态语义依赖。MIME超越多个单模态和多模态系统，在最佳基线上获得约4%的F1绝对提升。最后，我们对MIME性能进行详细分析，揭示了实现跨模态上下文关联最优建模的关键因素。