In the human brain, internal states are often correlated over time (due to local recurrence and other intrinsic circuit properties), punctuated by abrupt transitions. At first glance, temporal smoothness of internal states presents a problem for learning input-output mappings (e.g. category labels for images), because the internal representation of the input will contain a mixture of current input and prior inputs. However, when training with naturalistic data (e.g. movies) there is also temporal autocorrelation in the input. How does the temporal "smoothness" of internal states affect the efficiency of learning when the training data are also temporally smooth? How does it affect the kinds of representations that are learned? We found that, when trained with temporally smooth data, "slow" neural networks (equipped with linear recurrence and gating mechanisms) learned to categorize more efficiently than feedforward networks. Furthermore, networks with linear recurrence and multi-timescale gating could learn internal representations that "un-mixed" quickly-varying and slowly-varying data sources. Together, these findings demonstrate how a fundamental property of cortical dynamics (their temporal autocorrelation) can serve as an inductive bias, leading to more efficient category learning and to the representational separation of fast and slow sources in the environment.

翻译：在人脑中，内部状态通常随时间呈现相关性（由于局部循环及其他固有回路特性），并伴有突发性转换。表面看来，内部状态的时序平滑性对学习输入-输出映射（如图像类别标签）构成挑战，因为输入的内部表征会混杂当前输入与先前输入。然而，当使用自然istic数据（如电影）进行训练时，输入同样存在时序自相关性。当训练数据也具有时序平滑性时，内部状态的时序"平滑性"如何影响学习效率？它又如何影响习得的表征类型？我们发现，使用时序平滑数据训练时，配备线性循环与门控机制的"慢速"神经网络能比前馈网络更高效地学习分类。此外，具备线性循环与多时间尺度门控的网络能学习将快速变化与慢速变化数据源"解混"的内部表征。这些发现共同表明：皮层动态的基本属性（其时序自相关性）可充当归纳偏置，从而促进更高效的类别学习及环境中快速与慢速源的表征分离。