Reasoning over spans of tokens from different parts of the input is essential for natural language understanding (NLU) tasks such as fact-checking (FC), machine reading comprehension (MRC) or natural language inference (NLI). However, existing highlight-based explanations primarily focus on identifying individual important tokens or interactions only between adjacent tokens or tuples of tokens. Most notably, there is a lack of annotations capturing the human decision-making process w.r.t. the necessary interactions for informed decision-making in such tasks. To bridge this gap, we introduce SpanEx, a multi-annotator dataset of human span interaction explanations for two NLU tasks: NLI and FC. We then investigate the decision-making processes of multiple fine-tuned large language models in terms of the employed connections between spans in separate parts of the input and compare them to the human reasoning processes. Finally, we present a novel community detection based unsupervised method to extract such interaction explanations from a model's inner workings.

翻译：推理输入不同部分的令牌跨度对于自然语言理解（NLU）任务（如事实核查（FC）、机器阅读理解（MRC）或自然语言推理（NLI））至关重要。然而，现有的基于高亮的解释主要关注识别单个重要令牌，或仅关注相邻令牌或元组之间的交互。最值得注意的是，缺乏捕捉人类在这些任务中进行知情决策所需交互的决策过程的标注。为弥补这一空白，我们引入了SpanEx，一个针对两项NLU任务（NLI和FC）的人类跨度交互解释的多标注者数据集。接着，我们研究了多个微调大语言模型在输入不同部分之间使用的跨度连接方面的决策过程，并将其与人类推理过程进行比较。最后，我们提出了一种基于社区检测的全新无监督方法，用于从模型内部机制中提取此类交互解释。