Social media platforms serve as accessible outlets for individuals to express their thoughts and experiences, resulting in an influx of user-generated data spanning all age groups. While these platforms enable free expression, they also present significant challenges, including the proliferation of hate speech and offensive content. Such objectionable language disrupts objective discourse and can lead to radicalization of debates, ultimately threatening democratic values. Consequently, organizations have taken steps to monitor and curb abusive behavior, necessitating automated methods for identifying suspicious posts. This paper contributes to Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages (HASOC) 2023 shared tasks track. We, team Z-AGI Labs, conduct a comprehensive comparative analysis of hate speech classification across five distinct languages: Bengali, Assamese, Bodo, Sinhala, and Gujarati. Our study encompasses a wide range of pre-trained models, including Bert variants, XLM-R, and LSTM models, to assess their performance in identifying hate speech across these languages. Results reveal intriguing variations in model performance. Notably, Bert Base Multilingual Cased emerges as a strong performer across languages, achieving an F1 score of 0.67027 for Bengali and 0.70525 for Assamese. At the same time, it significantly outperforms other models with an impressive F1 score of 0.83009 for Bodo. In Sinhala, XLM-R stands out with an F1 score of 0.83493, whereas for Gujarati, a custom LSTM-based model outshined with an F1 score of 0.76601. This study offers valuable insights into the suitability of various pre-trained models for hate speech detection in multilingual settings. By considering the nuances of each, our research contributes to an informed model selection for building robust hate speech detection systems.

翻译：社交媒体平台为个人表达思想和经历提供了便捷渠道，导致跨年龄段的用户生成数据激增。这些平台虽允许自由表达，但也带来严峻挑战，包括仇恨言论与攻击性内容的蔓延。此类不当语言会干扰客观讨论，导致辩论激进化，最终威胁民主价值观。因此，各组织已采取措施监控和遏制攻击性行为，亟需自动化方法识别可疑帖子。本文针对2023年英语及印度-雅利安语言仇恨言论与攻击性内容识别（HASOC）共享任务展开研究。我们Z-AGI Labs团队对五种不同语言（孟加拉语、阿萨姆语、博多语、僧伽罗语和古吉拉特语）的仇恨言论分类进行了全面的比较分析。研究涵盖多种预训练模型，包括BERT变体、XLM-R和LSTM模型，以评估其在上述语言中识别仇恨言论的性能。结果揭示了模型性能的有趣差异：值得注意的是，BERT Base Multilingual Cased在多种语言中表现优异，在孟加拉语和阿萨姆语中分别取得0.67027和0.70525的F1分数，同时在博多语中以0.83009的F1分数显著超越其他模型；在僧伽罗语中，XLM-R以0.83493的F1分数脱颖而出；而在古吉拉特语中，基于自定义LSTM的模型以0.76601的F1分数表现最佳。本研究为多语言环境下仇恨言论检测中各类预训练模型的适用性提供了宝贵见解。通过考量每种语言的细微差异，我们的研究有助于为构建鲁棒的仇恨言论检测系统提供明智的模型选择依据。