Counter Narratives (CNs) are non-negative textual responses to Hate Speech (HS) aiming at defusing online hatred and mitigating its spreading across media. Despite the recent increase in HS content posted online, research on automatic CN generation has been relatively scarce and predominantly focused on English. In this paper, we present CONAN-EUS, a new Basque and Spanish dataset for CN generation developed by means of Machine Translation (MT) and professional post-edition. Being a parallel corpus, also with respect to the original English CONAN, it allows to perform novel research on multilingual and crosslingual automatic generation of CNs. Our experiments on CN generation with mT5, a multilingual encoder-decoder model, show that generation greatly benefits from training on post-edited data, as opposed to relying on silver MT data only. These results are confirmed by their correlation with a qualitative manual evaluation, demonstrating that manually revised training data remains crucial for the quality of the generated CNs. Furthermore, multilingual data augmentation improves results over monolingual settings for structurally similar languages such as English and Spanish, while being detrimental for Basque, a language isolate. Similar findings occur in zero-shot crosslingual evaluations, where model transfer (fine-tuning in English and generating in a different target language) outperforms fine-tuning mT5 on machine translated data for Spanish but not for Basque. This provides an interesting insight into the asymmetry in the multilinguality of generative models, a challenging topic which is still open to research.
翻译:反叙事(CNs)是针对仇恨言论(HS)的非负面文本回应,旨在消解网络仇恨并减缓其在媒体中的传播。尽管近年来在线发布的仇恨言论内容有所增加,但关于自动反叙事生成的研究相对匮乏,且主要集中于英语。本文提出了CONAN-EUS,一个通过机器翻译(MT)和专业后编辑开发的巴斯克语及西班牙语反叙事生成新数据集。作为与原英语CONAN平行的语料库,该数据集支持在多语言和跨语言自动反叙事生成领域开展创新研究。我们基于多语言编码器-解码器模型mT5进行的反叙事生成实验表明,相比于仅依赖银标准机器翻译数据,使用后编辑数据进行训练能显著提升生成质量。这一结果通过定性人工评估的相关性得到证实,说明人工修订的训练数据对生成反叙事的质量仍至关重要。此外,多语言数据增强能提升结构相似语言(如英语与西班牙语)的单语设置效果,但对孤立语言巴斯克语产生负面影响。在零样本跨语言评估中亦观察到类似发现:模型迁移(以英语微调后在目标语言生成)对西班牙语的表现优于使用机器翻译数据微调mT5,但对巴斯克语则不然。这为生成模型多语言性的非对称性提供了有趣见解——这一颇具挑战的课题仍有待研究。