Most evaluations of attribution methods focus on the English language. In this work, we present a multilingual approach for evaluating attribution methods for the Natural Language Inference (NLI) task in terms of faithfulness and plausibility. First, we introduce a novel cross-lingual strategy to measure faithfulness based on word alignments, which eliminates the drawbacks of erasure-based evaluations.We then perform a comprehensive evaluation of attribution methods, considering different output mechanisms and aggregation methods. Finally, we augment the XNLI dataset with highlight-based explanations, providing a multilingual NLI dataset with highlights, to support future exNLP studies. Our results show that attribution methods performing best for plausibility and faithfulness are different.

翻译：当前多数归因方法的评估主要聚焦于英语。本研究提出一种多语言方法，用于评估自然语言推理（NLI）任务中归因方法在忠实性与合理性方面的表现。首先，我们引入一种基于词对齐的新型跨语言策略来衡量忠实性，从而消除基于擦除评估的缺陷。随后，我们对归因方法进行了全面评估，涉及不同的输出机制与聚合方法。最后，我们为XNLI数据集补充了基于高亮的解释，构建了一个包含高亮标注的多语言NLI数据集，以支持未来的可解释NLP研究。结果表明，在合理性与忠实性方面表现最佳的归因方法存在差异。