Neural population activity often exhibits regime-dependent non-stationarity in the form of switching dynamics. Learning accurate switching dynamical system models can reveal how behavior is encoded in neural activity. Existing switching approaches have primarily focused on learning models from a single neural modality, either continuous Gaussian signals or discrete Poisson signals. However, multiple neural modalities are often recorded simultaneously to measure different spatiotemporal scales of brain activity, and all these modalities can encode behavior. Moreover, regime labels are typically unavailable in training data, posing a significant challenge for learning models of regime-dependent switching dynamics. To address these challenges, we develop a novel unsupervised learning algorithm that learns the parameters of switching multiscale dynamical system models using only multiscale neural observations. We demonstrate our method using both simulations and two distinct experimental datasets with multimodal spike-LFP observations during different motor tasks. We find that our switching multiscale dynamical system models more accurately decode behavior than switching single-scale dynamical models, showing the success of multiscale neural fusion. Further, our models outperform stationary multiscale models, illustrating the importance of tracking regime-dependent non-stationarity in multimodal neural data. The developed unsupervised learning framework enables more accurate modeling of complex multiscale neural dynamics by leveraging information in multimodal recordings while incorporating regime switches. This approach holds promise for improving the performance and robustness of brain-computer interfaces over time and for advancing our understanding of the neural basis of behavior.

翻译：神经群体活动常表现出依赖于状态的非平稳性，体现为切换动力学。学习精确的切换动力学系统模型能够揭示行为如何编码于神经活动中。现有的切换方法主要集中于从单一神经模态（连续高斯信号或离散泊松信号）学习模型。然而，为测量脑活动的不同时空尺度，常同时记录多种神经模态，且所有这些模态均可编码行为。此外，训练数据中通常缺乏状态标签，这为学习依赖于状态的切换动力学模型带来了显著挑战。为应对这些挑战，我们开发了一种新颖的无监督学习算法，仅利用多尺度神经观测数据学习切换多尺度动力学系统模型的参数。我们通过仿真及两个不同运动任务期间的多模态尖峰-局部场电位观测实验数据集验证了该方法。研究发现，相较于切换单尺度动力学模型，我们的切换多尺度动力学系统模型能更准确地解码行为，体现了多尺度神经融合的有效性。此外，我们的模型优于平稳多尺度模型，说明了在追踪多模态神经数据中依赖于状态的非平稳性的重要性。所开发的无监督学习框架通过融合多模态记录中的信息并纳入状态切换，实现了对复杂多尺度神经动力学的更精确建模。该方法有望随时间提升脑机接口的性能与鲁棒性，并推动对行为神经基础的理解。