Human cognition integrates information across nested timescales. While the cortex exhibits hierarchical Temporal Receptive Windows (TRWs), local circuits often display heterogeneous time constants. To reconcile this, we trained biologically constrained deep networks, based on scale-invariant hippocampal time cells, on a language classification task mimicking the hierarchical structure of language (e.g., 'letters' forming 'words'). First, using a feedforward model (SITHCon), we found that a hierarchy of TRWs emerged naturally across layers, despite the network having an identical spectrum of time constants within layers. We then distilled these inductive priors into a biologically plausible recurrent architecture, SITH-RNN. Training a sequence of architectures ranging from generic RNNs to this restricted subset showed that the scale-invariant SITH-RNN learned faster with orders-of-magnitude fewer parameters, and generalized zero-shot to out-of-distribution timescales. These results suggest the brain employs scale-invariant, sequential priors - coding "what" happened "when" - making recurrent networks with such priors particularly well-suited to describe human cognition.

翻译：人类认知整合了嵌套时间尺度上的信息。尽管皮层表现出分层的时间感受野（TRWs），但局部回路通常表现出异质的时间常数。为了调和这一矛盾，我们基于尺度不变的海马时间细胞，在模拟语言层次结构（例如“字母”构成“单词”）的语言分类任务上训练了生物约束深度网络。首先，使用前馈模型（SITHCon），我们发现尽管网络在层内具有相同的时间常数谱，但TRWs的层次结构在各层中自然涌现。随后，我们将这些归纳先验提炼成一个生物合理的循环架构，即SITH-RNN。通过训练一系列从通用RNN到这一受限子集的架构，结果表明尺度不变的SITH-RNN学习速度更快，且参数数量减少数个数量级，并能零样本泛化到分布外的时间尺度。这些结果表明，大脑采用了尺度不变的序列先验——编码“何时”发生“何事”——使得具备此类先验的循环网络特别适合描述人类认知。