Our brain can almost effortlessly decompose visual data streams into background and salient objects. Moreover, it can anticipate object motion and interactions, which are crucial abilities for conceptual planning and reasoning. Recent object reasoning datasets, such as CATER, have revealed fundamental shortcomings of current vision-based AI systems, particularly when targeting explicit object representations, object permanence, and object reasoning. Here we introduce a self-supervised LOCation and Identity tracking system (Loci), which excels on the CATER tracking challenge. Inspired by the dorsal and ventral pathways in the brain, Loci tackles the binding problem by processing separate, slot-wise encodings of `what' and `where'. Loci's predictive coding-like processing encourages active error minimization, such that individual slots tend to encode individual objects. Interactions between objects and object dynamics are processed in the disentangled latent space. Truncated backpropagation through time combined with forward eligibility accumulation significantly speeds up learning and improves memory efficiency. Besides exhibiting superior performance in current benchmarks, Loci effectively extracts objects from video streams and separates them into location and Gestalt components. We believe that this separation offers a representation that will facilitate effective planning and reasoning on conceptual levels.

翻译：我们的大脑能够几乎毫不费力地将视觉数据流分解为背景和显著物体。此外，它还能预测物体运动与交互，这些是概念规划与推理的关键能力。当前基于视觉的AI系统在面向显式物体表征、物体恒常性及物体推理时，暴露出根本性缺陷，例如最近的物体推理数据集CATER（组合注意力变换器评估基准）所揭示的问题。本文提出一种自监督的位置与身份追踪系统Loci，它在CATER追踪挑战中表现优异。受大脑背侧和腹侧通路启发，Loci通过处理“什么”和“在哪里”的分离式槽位编码来解决绑定问题。Loci的预测编码式处理鼓励主动误差最小化，使得单个槽位倾向于编码单个物体。物体间的交互和物体动力学在解耦的潜在空间中进行处理。结合前向资格累积的截断时间反向传播显著加速学习并提高记忆效率。除了在当前基准测试中展现优越性能外，Loci还能有效从视频流中提取物体，并将其分解为位置和格式塔成分。我们相信这种分离提供了一种表征，将有助于在概念层面进行有效的规划与推理。