With the increasing diversity of use cases of large language models, a more informative treatment of texts seems necessary. An argumentative analysis could foster a more reasoned usage of chatbots, text completion mechanisms or other applications. However, it is unclear which aspects of argumentation can be reliably identified and integrated in language models. In this paper, we present an empirical assessment of the reliability with which different argumentative aspects can be automatically identified in hate speech in social media. We have enriched the Hateval corpus (Basile et al. 2019) with a manual annotation of some argumentative components, adapted from Wagemans (2016)'s Periodic Table of Arguments. We show that some components can be identified with reasonable reliability. For those that present a high error ratio, we analyze the patterns of disagreement between expert annotators and errors in automatic procedures, and we propose adaptations of those categories that can be more reliably reproduced.

翻译：随着大语言模型应用场景的日益多样化，对文本进行更具信息量的处理显得尤为必要。论证分析能够促进对聊天机器人、文本补全机制或其他应用更合理的使用。然而，目前尚不明确哪些论证维度可被可靠识别并整合至语言模型中。本文对社交媒体仇恨言论中不同论证维度的自动识别可靠性进行了实证评估。我们基于Wagemans（2016）的《论证周期表》中的论证要素，对Hateval语料库（Basile等人，2019）进行了人工标注扩展。研究结果表明，部分论证要素能够以合理精度被识别。针对误差率较高的要素，我们分析了专家标注者间的分歧模式及自动识别流程中的错误，并对这些类别提出了更易可靠复现的适应性改进方案。