Internet Memes remain a challenging form of user-generated content for automated sentiment classification. The availability of labelled memes is a barrier to developing sentiment classifiers of multimodal memes. To address the shortage of labelled memes, we propose to supplement the training of a multimodal meme classifier with unimodal (image-only and text-only) data. In this work, we present a novel variant of supervised intermediate training that uses relatively abundant sentiment-labelled unimodal data. Our results show a statistically significant performance improvement from the incorporation of unimodal text data. Furthermore, we show that the training set of labelled memes can be reduced by 40% without reducing the performance of the downstream model.

翻译：互联网梗图仍然是自动化情感分类中具有挑战性的用户生成内容形式。带标签梗图的稀缺性阻碍了多模态梗图情感分类器的开发。为了解决带标签梗图不足的问题，我们提出利用单模态（仅图像和仅文本）数据来补充多模态梗图分类器的训练。在本工作中，我们提出了一种新颖的监督式中间训练变体，该变体利用相对丰富的情感标注单模态数据。实验结果表明，融入单模态文本数据后，模型性能实现了统计显著的提升。此外，我们证明标签梗图的训练集可减少40%，而下游模型性能不会降低。