The vital role of analogical reasoning in human cognition allows us to grasp novel concepts by linking them with familiar ones through shared relational structures. Despite the attention previous research has given to word analogies, this work suggests that Large Language Models (LLMs) often overlook the structures that underpin these analogies, raising questions about the efficacy of word analogies as a measure of analogical reasoning skills akin to human cognition. In response to this, our paper introduces a task of analogical structure abduction, grounded in cognitive psychology, designed to abduce structures that form an analogy between two systems. In support of this task, we establish a benchmark called SCAR, containing 400 scientific analogies from 13 distinct fields, tailored for evaluating analogical reasoning with structure abduction. The empirical evidence underlines the continued challenges faced by LLMs, including ChatGPT and GPT-4, in mastering this task, signifying the need for future exploration to enhance their abilities.

翻译：类比推理在人类认知中发挥着至关重要的作用，它通过共享的关系结构将新颖概念与熟悉概念联系起来，使我们能够理解新概念。尽管先前研究对词语类比给予了关注，但本研究指出，大型语言模型（LLMs）往往忽略了构成这些类比的结构，从而质疑了词语类比作为衡量类似于人类认知的类比推理能力的有效性。为此，我们的论文引入了一项基于认知心理学的类比结构外推任务，旨在推演出形成两个系统间类比的逻辑结构。为支持这一任务，我们建立了一个名为SCAR的基准测试集，包含来自13个不同领域的400个科学类比，专门用于评估带有结构外推能力的类比推理。实证证据表明，包括ChatGPT和GPT-4在内的大型语言模型在这项任务中仍面临持续挑战，这预示着未来需要进一步探索以提升其能力。