Based on the predictive map theory of spatial learning in animals, this study delves into the dynamics of Successor Feature (SF) and Predecessor Feature (PF) algorithms within noisy environments. Utilizing Q-learning and Q($\lambda$) learning as benchmarks for comparative analysis, our investigation yielded unexpected outcomes. Contrary to prevailing expectations and previous literature where PF demonstrated superior performance, our findings reveal that in noisy environments, PF did not surpass SF. In a one-dimensional grid world, SF exhibited superior adaptability, maintaining robust performance across varying noise levels. This trend of diminishing performance with increasing noise was consistent across all examined algorithms, indicating a linear degradation pattern. The scenario shifted in a two-dimensional grid world, where the impact of noise on algorithm performance demonstrated a non-linear relationship, influenced by the $\lambda$ parameter of the eligibility trace. This complexity suggests that the interaction between noise and algorithm efficacy is tied to the environmental dimensionality and specific algorithmic parameters. Furthermore, this research contributes to the bridging discourse between computational neuroscience and reinforcement learning (RL), exploring the neurobiological parallels of SF and PF learning in spatial navigation. Despite the unforeseen performance trends, the findings enrich our comprehension of the strengths and weaknesses inherent in RL algorithms. This knowledge is pivotal for advancing applications in robotics, gaming AI, and autonomous vehicle navigation, underscoring the imperative for continued exploration into how RL algorithms process and learn from noisy inputs.

翻译：基于动物空间学习的预测地图理论，本研究深入探讨了噪声环境下后继特征（SF）与前驱特征（PF）算法的动态特性。以Q学习和Q($\lambda$)学习作为比较分析的基准方法，我们的研究得出了意料之外的结果。与先前文献中PF表现更优的普遍预期和结论相反，我们的实验发现，在噪声环境中PF并未超越SF。在一维网格世界中，SF表现出更强的适应性，在不同噪声水平下均保持稳健性能。所有被检验算法均呈现出随噪声增加性能递减的线性退化模式。而在二维网格世界中，情况发生转变：噪声对算法性能的影响呈现出非线性关系，且受到资格迹$\lambda$参数的调节。这种复杂性表明，噪声与算法效能之间的相互作用与环境维度及特定算法参数密切相关。此外，本研究通过探索空间导航中SF与PF学习的神经生物学对应机制，为计算神经科学与强化学习（RL）之间的桥梁对话做出贡献。尽管性能趋势出人意料，但研究结果丰富了我们对RL算法内在优势与局限的理解。这一认知对于推进机器人、游戏AI及自动驾驶导航领域的应用至关重要，突出了持续探索RL算法如何处理噪声输入并从噪声中学习的必要性。