This paper presents a novel approach to address the challenge of online hidden representation learning for decision-making under uncertainty in non-stationary, partially observable environments. The proposed algorithm, Distributed Hebbian Temporal Memory (DHTM), is based on factor graph formalism and a multicomponent neuron model. DHTM aims to capture sequential data relationships and make cumulative predictions about future observations, forming Successor Representation (SR). Inspired by neurophysiological models of the neocortex, the algorithm utilizes distributed representations, sparse transition matrices, and local Hebbian-like learning rules to overcome the instability and slow learning process of traditional temporal memory algorithms like RNN and HMM. Experimental results demonstrate that DHTM outperforms classical LSTM and performs comparably to more advanced RNN-like algorithms, speeding up Temporal Difference learning for SR in changing environments. Additionally, we compare the SRs produced by DHTM to another biologically inspired HMM-like algorithm, CSCG. Our findings suggest that DHTM is a promising approach for addressing the challenges of online hidden representation learning in dynamic environments.

翻译：本文提出了一种新颖方法，以解决非平稳、部分可观测环境中在线隐藏表征学习在不确定性下进行决策的挑战。所提算法——分布式赫布时序记忆（DHTM），基于因子图形式化与多组分神经元模型。DHTM旨在捕获序列数据关系并形成对未来观测的累积预测，构建后继表征（SR）。受新皮层神经生理学模型启发，该算法利用分布式表征、稀疏转移矩阵及局部赫布型学习规则，克服了传统时序记忆算法（如RNN和HMM）的不稳定性与学习缓慢问题。实验结果表明，DHTM性能优于经典LSTM，且与更先进的RNN类算法相当，在动态环境中加速了SR的时序差分学习。此外，我们将DHTM生成的SR与另一类生物启发式HMM算法CSCG进行了比较。研究结果表明，DHTM是一种应对动态环境中在线隐藏表征学习挑战的有前景方法。