Natural Language Inference (NLI) datasets often exhibit human label variation. To better understand these variations, explanation-based approaches analyze the underlying reasoning behind annotators' decisions. One such approach is the LiTEx taxonomy, which categorizes free-text explanations in English into reasoning categories. However, previous work applying LiTEx has focused on within-label variation: cases where annotators agree on the NLI label but provide different explanations. This paper broadens the scope by examining how annotators may diverge not only in the reasoning category but also in the labeling. We use explanations as a lens to analyze variation in NLI annotations and to examine individual differences in reasoning. We apply LiTEx to two NLI datasets and align annotation variation from multiple aspects: NLI label agreement, explanation similarity, and taxonomy agreement, with an additional compounding factor of annotators' selection bias. We observe instances where annotators disagree on the label but provide similar explanations, suggesting that surface-level disagreement may mask underlying agreement in interpretation. Moreover, our analysis reveals individual preferences in explanation strategies and label choices. These findings highlight that agreement in reasoning categories better reflects the semantic similarity of explanations than label agreement alone. Our findings underscore the richness of reasoning-based explanations and the need for caution in treating labels as ground truth.

翻译：自然语言推理（NLI）数据集常表现出人类标注差异。为更深入理解这些差异，基于解释的方法分析标注者决策背后的潜在推理过程。LiTEx分类法便是此类方法之一，它将英语自由文本解释归类为不同推理类别。然而，此前应用LiTEx的研究聚焦于标签内差异：即标注者对NLI标签达成一致但提供了不同解释的案例。本文通过考察标注者不仅在推理类别上存在分歧，在标签标注层面也可能产生差异，从而拓宽了研究范围。我们以解释为透镜，分析NLI标注中的差异并探究个体推理差异。我们将LiTEx应用于两个NLI数据集，从多维度对齐标注差异：NLI标签一致性、解释相似性、分类法一致性，并额外纳入标注者选择偏差这一复合因素。我们观察到标注者在标签上存在分歧但提供相似解释的案例，这表明表面标签分歧可能掩盖了潜在的理解一致性。此外，我们的分析揭示了标注者在解释策略和标签选择上的个体偏好。这些发现表明，推理类别的一致性相较于标签一致性更能反映解释的语义相似性。研究结果凸显了基于推理的解释的丰富性，并警示需谨慎对待将标签视为真实标注的做法。