Mental disorders present challenges in diagnosis and treatment due to their complex and heterogeneous nature. Electroencephalogram (EEG) has shown promise as a potential biomarker for these disorders. However, existing methods for analyzing EEG signals have limitations in addressing heterogeneity and capturing complex brain activity patterns between regions. This paper proposes a novel random effects state-space model (RESSM) for analyzing large-scale multi-channel resting-state EEG signals, accounting for the heterogeneity of brain connectivities between groups and individual subjects. We incorporate multi-level random effects for temporal dynamical and spatial mapping matrices and address nonstationarity so that the brain connectivity patterns can vary over time. The model is fitted under a Bayesian hierarchical model framework coupled with a Gibbs sampler. Compared to previous mixed-effects state-space models, we directly model high-dimensional random effects matrices without structural constraints and tackle the challenge of identifiability. Through extensive simulation studies, we demonstrate that our approach yields valid estimation and inference. We apply RESSM to a multi-site clinical trial of Major Depressive Disorder (MDD). Our analysis uncovers significant differences in resting-state brain temporal dynamics among MDD patients compared to healthy individuals. In addition, we show the subject-level EEG features derived from RESSM exhibit a superior predictive value for the heterogeneous treatment effect compared to the EEG frequency band power, suggesting the potential of EEG as a valuable biomarker for MDD.

翻译：精神障碍因其复杂性和异质性在诊断和治疗中面临挑战。脑电图作为这些障碍的潜在生物标志物已展现出希望。然而，现有分析脑电信号的方法在应对异质性和捕捉区域间复杂大脑活动模式方面存在局限性。本文提出一种新型随机效应状态空间模型，用于分析大规模多通道静息态脑电信号，并考虑群体和个体受试者之间大脑连接性的异质性。我们引入多级随机效应对时间动态和空间映射矩阵进行建模，并解决非平稳性问题，从而使大脑连接模式能够随时间变化。该模型在贝叶斯分层模型框架下结合吉布斯采样器进行拟合。与之前的混合效应状态空间模型相比，我们直接对高维随机效应矩阵进行建模，无需结构约束，并解决了可识别性难题。通过广泛的模拟研究，我们证明了所提出方法能产生有效的估计和推断。我们将该模型应用于一项多中心重度抑郁症临床试验。分析揭示了重度抑郁症患者与健康个体在静息态大脑时间动态上的显著差异。此外，我们表明基于该模型提取的受试者级别脑电图特征，在预测异质性治疗效果方面优于脑电图频带功率，这提示了脑电图作为重度抑郁症有价值生物标志物的潜力。