The dramatic increase in the use of social media platforms for information sharing has also fueled a steep growth in online abuse. A simple yet effective way of abusing individuals or communities is by creating memes, which often integrate an image with a short piece of text layered on top of it. Such harmful elements are in rampant use and are a threat to online safety. Hence it is necessary to develop efficient models to detect and flag abusive memes. The problem becomes more challenging in a low-resource setting (e.g., Bengali memes, i.e., images with Bengali text embedded on it) because of the absence of benchmark datasets on which AI models could be trained. In this paper we bridge this gap by building a Bengali meme dataset. To setup an effective benchmark we implement several baseline models for classifying abusive memes using this dataset. We observe that multimodal models that use both textual and visual information outperform unimodal models. Our best-performing model achieves a macro F1 score of 70.51. Finally, we perform a qualitative error analysis of the misclassified memes of the best-performing text-based, image-based and multimodal models.

翻译：社交媒体平台在信息分享中的广泛应用，也加剧了网络欺凌行为的激增。一种简单而有效的欺凌个人或社区的方式是制作模因——通常将图像与叠加在其上的简短文字相结合。此类有害元素被大量使用，对网络环境构成威胁。因此，开发能够检测并标记恶意模因的高效模型至关重要。在低资源情境下（例如孟加拉语模因，即嵌有孟加拉语文字的图像），这一挑战更为严峻，因为缺乏可供AI模型训练的基准数据集。本文通过构建孟加拉语模因数据集填补了这一空白。为建立有效的基准，我们利用该数据集实现了多个基线模型用于恶意模因分类。研究发现，同时利用文本和视觉信息的多模态模型优于单模态模型。我们的最优模型在宏F1分数上达到70.51。最后，我们对最优的基于文本、基于图像以及多模态模型中误分类的模因进行了定性错误分析。