Language models' ability to extrapolate learned behaviors to novel, more complex environments beyond their training scope is highly unknown. This study introduces a path planning task in a textualized Gridworld to probe language models' extrapolation capabilities. We show that conventional approaches, including next token prediction and Chain of Thought (CoT) finetuning, fail to extrapolate in larger, unseen environments. Inspired by human cognition and dual process theory, we propose cognitive maps for path planning, a novel CoT framework that simulates humanlike mental representations. Our experiments show that cognitive maps not only enhance extrapolation to unseen environments but also exhibit humanlike characteristics through structured mental simulation and rapid adaptation. Our finding that these cognitive maps require specialized training schemes and cannot be induced through simple prompting opens up important questions about developing general-purpose cognitive maps in language models. Our comparison with exploration-based methods further illuminates the complementary strengths of offline planning and online exploration.

翻译：语言模型将习得行为外推至训练范围之外新颖、更复杂环境的能力尚不明确。本研究通过在文本化网格世界中引入路径规划任务，探究语言模型的外推能力。我们发现包括下一词预测和思维链微调在内的传统方法在更大、未见过的环境中均无法实现有效外推。受人类认知与双过程理论启发，我们提出用于路径规划的认知地图——一种模拟人类心智表征的新型思维链框架。实验表明，认知地图不仅能增强对未见环境的外推能力，还通过结构化心理模拟和快速适应展现出类人特性。我们发现这些认知地图需要专门训练方案且无法通过简单提示诱导，这为开发语言模型中通用认知地图提出了重要问题。与基于探索方法的对比进一步揭示了离线规划与在线探索的互补优势。