Artificial neural networks that can recover latent dynamics from recorded neural activity may provide a powerful avenue for identifying and interpreting the dynamical motifs underlying biological computation. Given that neural variance alone does not uniquely determine a latent dynamical system, interpretable architectures should prioritize accurate and low-dimensional latent dynamics. In this work, we evaluated the performance of sequential autoencoders (SAEs) in recovering latent chaotic attractors from simulated neural datasets. We found that SAEs with widely-used recurrent neural network (RNN)-based dynamics were unable to infer accurate firing rates at the true latent state dimensionality, and that larger RNNs relied upon dynamical features not present in the data. On the other hand, SAEs with neural ordinary differential equation (NODE)-based dynamics inferred accurate rates at the true latent state dimensionality, while also recovering latent trajectories and fixed point structure. Ablations reveal that this is mainly because NODEs (1) allow use of higher-capacity multi-layer perceptrons (MLPs) to model the vector field and (2) predict the derivative rather than the next state. Decoupling the capacity of the dynamics model from its latent dimensionality enables NODEs to learn the requisite low-D dynamics where RNN cells fail. Additionally, the fact that the NODE predicts derivatives imposes a useful autoregressive prior on the latent states. The suboptimal interpretability of widely-used RNN-based dynamics may motivate substitution for alternative architectures, such as NODE, that enable learning of accurate dynamics in low-dimensional latent spaces.

翻译：能够从记录的神经活动中恢复潜在动力学的人工神经网络，为识别和解释生物计算背后的动力学基序提供了强大途径。鉴于神经方差本身无法唯一确定潜在动力系统，可解释架构应优先关注准确且低维的潜在动力学。在本工作中，我们评估了序列自编码器（SAEs）从模拟神经数据集中恢复潜在混沌吸引子的性能。研究发现，采用广泛使用的基于循环神经网络（RNN）动力学的SAE无法在真实潜在状态维度下推断出准确的发放率，且更大的RNN依赖于数据中不存在的动力学特征。另一方面，采用基于神经常微分方程（NODE）动力学的SAE能在真实潜在状态维度下推断出准确发放率，同时还能恢复潜在轨迹和不动点结构。消融实验表明，这主要因为NODE（1）允许使用更高容量的多层感知机（MLP）建模向量场，以及（2）预测导数而非下一状态。将动力学模型的容量与其潜在维度解耦，使得NODE能在RNN细胞失败的情况下学习所需的低维动力学。此外，NODE预测导数这一特性对潜在状态施加了有用的自回归先验。广泛使用的基于RNN动力学的次优可解释性，可能推动其被NODE等替代架构取代，从而在低维潜在空间中实现精确动力学的学习。