Artificial neural networks that can recover latent dynamics from recorded neural activity may provide a powerful avenue for identifying and interpreting the dynamical motifs underlying biological computation. Given that neural variance alone does not uniquely determine a latent dynamical system, interpretable architectures should prioritize accurate and low-dimensional latent dynamics. In this work, we evaluated the performance of sequential autoencoders (SAEs) in recovering latent chaotic attractors from simulated neural datasets. We found that SAEs with widely-used recurrent neural network (RNN)-based dynamics were unable to infer accurate firing rates at the true latent state dimensionality, and that larger RNNs relied upon dynamical features not present in the data. On the other hand, SAEs with neural ordinary differential equation (NODE)-based dynamics inferred accurate rates at the true latent state dimensionality, while also recovering latent trajectories and fixed point structure. Ablations reveal that this is mainly because NODEs (1) allow use of higher-capacity multi-layer perceptrons (MLPs) to model the vector field and (2) predict the derivative rather than the next state. Decoupling the capacity of the dynamics model from its latent dimensionality enables NODEs to learn the requisite low-D dynamics where RNN cells fail. Additionally, the fact that the NODE predicts derivatives imposes a useful autoregressive prior on the latent states. The suboptimal interpretability of widely-used RNN-based dynamics may motivate substitution for alternative architectures, such as NODE, that enable learning of accurate dynamics in low-dimensional latent spaces.

翻译：人工神经网络能够从记录的神经活动中恢复潜在动力学，这为识别和解释生物计算背后的动力学模式提供了有力途径。鉴于神经方差本身无法唯一确定潜在动力系统，可解释架构应优先考虑准确且低维的潜在动力学。本研究评估了序列自编码器（SAEs）从模拟神经数据集中恢复潜在混沌吸引子的性能。研究发现，采用广泛使用的基于循环神经网络（RNN）动力学的SAEs无法在真实潜在状态维度下推断准确的发放率，且更大规模的RNNs依赖数据中不存在的动力学特征。相比之下，采用基于神经常微分方程（NODE）动力学的SAEs在真实潜在状态维度下推断出准确的发放率，同时还能恢复潜在轨迹和不动点结构。消融实验表明，这主要归因于NODEs：（1）允许使用更高容量的多层感知器（MLPs）建模向量场；（2）预测导数而非下一个状态。将动力学模型容量与其潜在维度解耦，使NODEs能够学习RNN细胞无法实现的低维动力学。此外，NODE预测导数这一特性为潜在状态施加了有用的自回归先验。基于广泛使用的RNN动力学的次优可解释性，可能推动替代架构（如NODE）的应用，这些架构能在低维潜在空间中学习准确动力学。