The focus of this study is to investigate the impact of different initialization strategies for the weight matrix of Successor Features (SF) on learning efficiency and convergence in Reinforcement Learning (RL) agents. Using a grid-world paradigm, we compare the performance of RL agents, whose SF weight matrix is initialized with either an identity matrix, zero matrix, or a randomly generated matrix (using Xavier, He, or uniform distribution method). Our analysis revolves around evaluating metrics such as value error, step length, PCA of Successor Representation (SR) place field, and the distance of SR matrices between different agents. The results demonstrate that RL agents initialized with random matrices reach the optimal SR place field faster and showcase a quicker reduction in value error, pointing to more efficient learning. Furthermore, these random agents also exhibit a faster decrease in step length across larger grid-world environments. The study provides insights into the neurobiological interpretations of these results, their implications for understanding intelligence, and potential future research directions. These findings could have profound implications for the field of artificial intelligence, particularly in the design of learning algorithms.

翻译：本研究的重点在于探讨强化学习（RL）智能体中后继特征（SF）权重矩阵的不同初始化策略对学习效率和收敛性的影响。利用网格世界范式，我们比较了将SF权重矩阵初始化为单位矩阵、零矩阵或随机生成矩阵（采用Xavier、He或均匀分布方法）的RL智能体的性能。我们的分析围绕评估价值误差、步长、后继表征（SR）位置场的PCA以及不同智能体之间SR矩阵的距离等指标展开。结果表明，初始化随机矩阵的RL智能体能更快达到最优SR位置场，并显示出价值误差的更快降低，这表明其学习效率更高。此外，在更大的网格世界环境中，这些随机初始化智能体也表现出步长的更快下降。本研究提供了这些结果的神经生物学解释、其对理解智能的意义以及潜在未来研究方向的洞见。这些发现可能对人工智能领域，特别是在学习算法设计方面，产生深远影响。