Hate speech detection is a challenging natural language processing task that requires capturing linguistic and contextual nuances. Pre-trained language models (PLMs) offer rich semantic representations of text that can improve this task. However there is still limited knowledge about ways to effectively combine representations across PLMs and leverage their complementary strengths. In this work, we shed light on various combination techniques for several PLMs and comprehensively analyze their effectiveness. Our findings show that combining embeddings leads to slight improvements but at a high computational cost and the choice of combination has marginal effect on the final outcome. We also make our codebase public at https://github.com/aflah02/The-Art-of-Embedding-Fusion-Optimizing-Hate-Speech-Detection .

翻译：仇恨言论检测是一项具有挑战性的自然语言处理任务，需要捕捉语言和上下文层面的细微差别。预训练语言模型（PLMs）能够提供文本的丰富语义表示，从而助力该任务的性能提升。然而，关于如何有效融合不同PLM的表示并利用其互补优势，目前仍缺乏深入认知。本研究系统阐释了多种PLM的表示组合技术，并对其有效性进行了全面分析。研究结果表明，嵌入融合虽能带来轻微改进，但计算成本高昂，且融合方式的选择对最终结果影响甚微。相关代码已开源发布于 https://github.com/aflah02/The-Art-of-Embedding-Fusion-Optimizing-Hate-Speech-Detection 。