Digital platforms have an ever-expanding user base, and act as a hub for communication, business, and connectivity. However, this has also allowed for the spread of hate speech and misogyny. Artificial intelligence models have emerged as an effective solution for countering online hate speech but are under explored for low resource and code-mixed languages and suffer from a lack of interpretability. Explainable Artificial Intelligence (XAI) can enhance transparency in the decisions of deep learning models, which is crucial for a sensitive domain such as hate speech detection. In this paper, we present a multi-modal and explainable web application for detecting misogyny in text and memes in code-mixed Hindi and English. The system leverages state-of-the-art transformer-based models that support multilingual and multimodal settings. For text-based misogyny identification, the system utilizes XLM-RoBERTa (XLM-R) and multilingual Bidirectional Encoder Representations from Transformers (mBERT) on a dataset of approximately 4,193 comments. For multimodal misogyny identification from memes, the system utilizes mBERT + EfficientNet, and mBERT + ResNET trained on a dataset of approximately 4,218 memes. It also provides feature importance scores using explainability techniques including Shapley Additive Values (SHAP) and Local Interpretable Model Agnostic Explanations (LIME). The application aims to serve as a tool for both researchers and content moderators, to promote further research in the field, combat gender based digital violence, and ensure a safe digital space. The system has been evaluated using human evaluators who provided their responses on Chatbot Usability Questionnaire (CUQ) and User Experience Questionnaire (UEQ) to determine overall usability.

翻译：数字平台拥有不断扩大的用户基础，并作为沟通、商业和连接的枢纽。然而，这也助长了仇恨言论和厌女症的传播。人工智能模型已成为应对在线仇恨言论的有效解决方案，但在低资源和代码混合语言方面尚未得到充分探索，且缺乏可解释性。可解释人工智能（XAI）可以增强深度学习模型决策的透明度，这对于仇恨言论检测等敏感领域至关重要。在本文中，我们提出了一种多模态且可解释的Web应用程序，用于检测印地语和英语混合代码的文本和表情包中的厌女症。该系统利用了支持多语言和多模态设置的、基于Transformer的最先进模型。对于基于文本的厌女症识别，该系统在包含约4,193条评论的数据集上使用了XLM-RoBERTa（XLM-R）和多语言Transformer双向编码器表示（mBERT）。对于从表情包中识别多模态厌女症，该系统使用了在约4,218个表情包数据集上训练的mBERT + EfficientNet和mBERT + ResNET。它还通过使用可解释性技术，包括沙普利加性解释（SHAP）和局部可解释模型无关解释（LIME），提供特征重要性分数。该应用程序旨在为研究人员和内容审核员提供一个工具，以促进该领域的进一步研究，打击基于性别的数字暴力，并确保安全的数字空间。该系统已通过人工评估员使用聊天机器人可用性问卷（CUQ）和用户体验问卷（UEQ）提供反馈进行了评估，以确定其整体可用性。