Qualitative Spatial Reasoning is a well explored area of Knowledge Representation and Reasoning and has multiple applications ranging from Geographical Information Systems to Robotics and Computer Vision. Recently, many claims have been made for the reasoning capabilities of Large Language Models (LLMs). Here, we investigate the extent to which a set of representative LLMs can perform classical qualitative spatial reasoning tasks on the mereotopological Region Connection Calculus, RCC-8. We conduct three pairs of experiments (reconstruction of composition tables, alignment to human composition preferences, conceptual neighbourhood reconstruction) using state-of-the-art LLMs; in each pair one experiment uses eponymous relations and one, anonymous relations (to test the extent to which the LLM relies on knowledge about the relation names obtained during training). All instances are repeated 30 times to measure the stochasticity of the LLMs.

翻译：定性空间推理是知识表示与推理领域的一个深入探索方向，其应用范围涵盖地理信息系统、机器人学和计算机视觉等多个领域。近来，关于大型语言模型（LLMs）推理能力的诸多论断不断涌现。本文旨在探究一组代表性LLMs在经典定性空间推理任务——基于区域连接演算（RCC-8）的拓扑关系推理——上的表现能力。我们采用最先进的LLMs进行了三组对照实验（组合表重构、与人类组合偏好对齐、概念邻域重构）；每组实验均包含使用具名关系和匿名关系两种条件（以检验LLM在多大程度上依赖训练过程中获得的关系名称知识）。所有实验实例均重复30次以测量LLMs的随机性。