Explainability methods are used to benchmark the extent to which model predictions align with human rationales i.e., are 'right for the right reasons'. Previous work has failed to acknowledge, however, that what counts as a rationale is sometimes subjective. This paper presents what we think is a first of its kind, a collection of human rationale annotations augmented with the annotators demographic information. We cover three datasets spanning sentiment analysis and common-sense reasoning, and six demographic groups (balanced across age and ethnicity). Such data enables us to ask both what demographics our predictions align with and whose reasoning patterns our models' rationales align with. We find systematic inter-group annotator disagreement and show how 16 Transformer-based models align better with rationales provided by certain demographic groups: We find that models are biased towards aligning best with older and/or white annotators. We zoom in on the effects of model size and model distillation, finding -- contrary to our expectations -- negative correlations between model size and rationale agreement as well as no evidence that either model size or model distillation improves fairness.

翻译：可解释性方法常用于评估模型预测与人类理据的匹配程度，即是否“基于正确理由得到正确结论”。然而，先前研究未能认识到，理据的判定有时具有主观性。本文首次提出一个包含标注者人口统计信息的人类理据标注集，覆盖情感分析与常识推理三个数据集，包含六个跨年龄与种族平衡的人口统计组别。该数据集使我们能够同时探究：我们的预测结果与哪些人口群体的观点相符，以及模型的推理模式与谁的推理模式一致。我们发现标注者之间存在系统性群体间分歧，并揭示16个基于Transformer的模型更倾向于与特定人口群体的理据保持一致：模型偏向于与年长和/或白人标注者的理据匹配度最高。我们进一步聚焦模型规模与蒸馏的影响，发现——与预期相反——模型规模与理据一致性呈负相关，且无证据表明模型规模或蒸馏能提升公平性。