Episodic memory plays a crucial role in various cognitive processes, such as the ability to mentally recall past events. While cognitive science emphasizes the significance of spatial context in the formation and retrieval of episodic memory, the current primary approach to implementing episodic memory in AI systems is through transformers that store temporally ordered experiences, which overlooks the spatial dimension. As a result, it is unclear how the underlying structure could be extended to incorporate the spatial axis beyond temporal order alone and thereby what benefits can be obtained. To address this, this paper explores the use of Spatially-Aware Transformer models that incorporate spatial information. These models enable the creation of place-centric episodic memory that considers both temporal and spatial dimensions. Adopting this approach, we demonstrate that memory utilization efficiency can be improved, leading to enhanced accuracy in various place-centric downstream tasks. Additionally, we propose the Adaptive Memory Allocator, a memory management method based on reinforcement learning that aims to optimize efficiency of memory utilization. Our experiments demonstrate the advantages of our proposed model in various environments and across multiple downstream tasks, including prediction, generation, reasoning, and reinforcement learning. The source code for our models and experiments will be available at https://github.com/junmokane/spatially-aware-transformer.

翻译：情景记忆在多种认知过程中发挥着关键作用，例如对过去事件的心理回忆能力。虽然认知科学强调空间上下文在情景记忆形成和提取中的重要性，但目前人工智能系统中实现情景记忆的主要方式是通过存储时间顺序经验的前馈神经Transformer，这忽略了空间维度。因此，现有结构如何超越单纯的时间顺序而整合空间轴，以及由此能获得何种益处尚不明确。为解决这一问题，本文探索了融合空间信息的空间感知Transformer模型。这些模型能够创建同时考虑时间和空间维度的场所中心情景记忆。采用该方法，我们证明了记忆利用效率可得到提升，从而在多种场所中心的下游任务中提高准确性。此外，我们提出了基于强化学习的记忆管理方法——自适应记忆分配器，旨在优化记忆利用效率。实验表明，我们提出的模型在多种环境及预测、生成、推理和强化学习等多个下游任务中均展现出优势。本文模型和实验的源代码将发布于https://github.com/junmokane/spatially-aware-transformer。