This paper presents the best-performing approach alias "Adam Smith" for the SemEval-2023 Task 4: "Identification of Human Values behind Arguments". The goal of the task was to create systems that automatically identify the values within textual arguments. We train transformer-based models until they reach their loss minimum or f1-score maximum. Ensembling the models by selecting one global decision threshold that maximizes the f1-score leads to the best-performing system in the competition. Ensembling based on stacking with logistic regressions shows the best performance on an additional dataset provided to evaluate the robustness ("Nahj al-Balagha"). Apart from outlining the submitted system, we demonstrate that the use of the large ensemble model is not necessary and that the system size can be significantly reduced.
翻译:本文介绍了在SemEval-2023任务4“论证背后人类价值观的识别”中表现最优的方法,即“Adam Smith”方法。该任务旨在构建能够自动识别文本论证中价值观的系统。我们训练基于Transformer的模型,直至其达到损失最小值或F1分数最大值。通过选取最大化F1分数的单一全局决策阈值来集成模型,从而在竞赛中获得了最佳性能的系统。基于逻辑回归堆叠的集成方法在另一个用于评估鲁棒性的数据集(《纳赫杰·巴拉赫》)上展现出最佳性能。除了概述所提交的系统外,我们还证明使用大型集成模型并非必要,系统规模可以显著缩减。