Analogical reasoning is a key driver of human generalization in problem-solving and argumentation. Yet, analogies between narrative structures remain challenging for machines. Cognitive engines for structural mapping are not directly applicable, as they assume pre-extracted entities, whereas LLMs' performance is sensitive to prompt format and the degree of surface similarity between narratives. This gap motivates a key question: What is the impact of enhancing structural mapping with LLM-derived abstractions on their analogical reasoning ability in narratives? To that end, we propose a modular framework named YARN (Yielding Abstractions for Reasoning in Narratives), which uses LLMs to decompose narratives into units, abstract these units, and then passes them to a mapping component that aligns elements across stories to perform analogical reasoning. We define and operationalize four levels of abstraction that capture both the general meaning of units and their roles in the story, grounded in prior work on framing. Our experiments reveal that abstractions consistently improve model performance, resulting in competitive or better performance than end-to-end LLM baselines. Closer error analysis reveals the remaining challenges in abstraction at the right level, in incorporating implicit causality, and an emerging categorization of analogical patterns in narratives. YARN enables systematic variation of experimental settings to analyze component contributions, and to support future work, we make the code for YARN openly available.
翻译:类比推理是人类在问题解决和论证中实现泛化的关键驱动力。然而,叙事结构之间的类比对于机器而言仍具挑战性。用于结构映射的认知引擎无法直接应用,因为它们假设实体已被预先提取,而大语言模型(LLM)的性能对提示格式和叙事间表面相似度高度敏感。这一差距引出了一个关键问题:将LLM生成的抽象用于增强结构映射,对机器在叙事中的类比推理能力有何影响?为此,我们提出一个模块化框架YARN(叙事推理的抽象生成框架),该框架利用LLM将叙事分解为单元,对这些单元进行抽象,随后将抽象结果传递给映射组件,该组件对齐不同故事中的元素以执行类比推理。我们基于先前关于框架的研究,定义并实现了四个抽象层级,这些层级既能捕捉单元的普遍含义,也能反映其在故事中的角色。实验表明,抽象一致地提升了模型性能,使其性能达到或优于端到端LLM基线。深入的错误分析揭示了在适当层级进行抽象、融入隐含因果关系方面存在的挑战,以及叙事中类比模式的新兴分类。YARN支持系统性地变化实验设置以分析各组件贡献,为支持未来研究,我们公开了YARN的代码。