In this study, we aimed to address the growing concern of trolling behavior on social media by developing and evaluating a set of model architectures for the automatic detection of troll tweets. Utilizing deep learning techniques and pre-trained word embedding methods such as BERT, ELMo, and GloVe, we evaluated the performance of each architecture using metrics such as classification accuracy, F1 score, AUC, and precision. Our results indicate that BERT and ELMo embedding methods performed better than the GloVe method, likely due to their ability to provide contextualized word embeddings that better capture the nuances and subtleties of language use in online social media. Additionally, we found that CNN and GRU encoders performed similarly in terms of F1 score and AUC, suggesting their effectiveness in extracting relevant information from input text. The best-performing method was found to be an ELMo-based architecture that employed a GRU classifier, with an AUC score of 0.929. This research highlights the importance of utilizing contextualized word embeddings and appropriate encoder methods in the task of troll tweet detection, which can assist social-based systems in improving their performance in identifying and addressing trolling behavior on their platforms.
翻译:本研究旨在应对社交媒体上日益严重的网络恶搞行为,通过开发并评估一系列用于自动检测恶搞推文的模型架构来解决问题。我们利用深度学习技术及BERT、ELMo、GloVe等预训练词嵌入方法,采用分类准确率、F1分数、AUC和精确率等指标对各架构性能进行评估。结果表明,BERT和ELMo嵌入方法的表现优于GloVe方法,这主要归因于其能够提供上下文相关词嵌入,从而更精准地捕捉在线社交媒体语言使用的细微特征。此外,我们发现CNN与GRU编码器在F1分数和AUC指标上表现相似,这表明两者在提取输入文本相关信息方面具有同等有效性。表现最优的模型是基于ELMo架构、采用GRU分类器的方案,其AUC得分达到0.929。本研究凸显了在恶搞推文检测任务中利用上下文词嵌入与合适编码方法的重要性,此类方法可帮助社交平台系统提升识别和应对网络恶搞行为的性能。