In modern neuroscience, functional magnetic resonance imaging (fMRI) has been a crucial and irreplaceable tool that provides a non-invasive window into the dynamics of whole-brain activity. Nevertheless, fMRI is limited by hemodynamic blurring as well as high cost, immobility, and incompatibility with metal implants. Electroencephalography (EEG) is complementary to fMRI and can directly record the cortical electrical activity at high temporal resolution, but has more limited spatial resolution and is unable to recover information about deep subcortical brain structures. The ability to obtain fMRI information from EEG would enable cost-effective, imaging across a wider set of brain regions. Further, beyond augmenting the capabilities of EEG, cross-modality models would facilitate the interpretation of fMRI signals. However, as both EEG and fMRI are high-dimensional and prone to artifacts, it is currently challenging to model fMRI from EEG. To address this challenge, we propose a novel architecture that can predict fMRI signals directly from multi-channel EEG without explicit feature engineering. Our model achieves this by implementing a Sinusoidal Representation Network (SIREN) to learn frequency information in brain dynamics from EEG, which serves as the input to a subsequent encoder-decoder to effectively reconstruct the fMRI signal from a specific brain region. We evaluate our model using a simultaneous EEG-fMRI dataset with 8 subjects and investigate its potential for predicting subcortical fMRI signals. The present results reveal that our model outperforms a recent state-of-the-art model, and indicates the potential of leveraging periodic activation functions in deep neural networks to model functional neuroimaging data.

翻译：在现代神经科学中，功能性磁共振成像（fMRI）一直是不可或缺的关键工具，它以非侵入性方式揭示全脑活动的动态变化。然而，fMRI受限于血流动力学模糊效应、高成本、设备不可移动性以及与金属植入物的不兼容性。脑电图（EEG）作为fMRI的补充技术，能够以高时间分辨率直接记录皮层电活动，但其空间分辨率有限，且无法获取深部皮层下脑结构的信息。从EEG中获取fMRI信息的能力将实现更具成本效益的跨脑区成像。此外，跨模态模型不仅能增强EEG的功能，还能促进对fMRI信号的理解。然而，由于EEG和fMRI均具有高维度且易受伪影干扰，当前从EEG建模fMRI仍面临挑战。为解决这一问题，我们提出了一种新型架构，无需显式特征工程即可直接从多通道EEG预测fMRI信号。该模型通过引入正弦表示网络（SIREN）学习EEG中脑动力学的频率信息，并将其作为后续编码器-解码器的输入，从而有效重建特定脑区的fMRI信号。我们利用包含8名受试者的同步EEG-fMRI数据集评估模型性能，并探索其预测皮层下fMRI信号的潜力。结果表明，我们的模型优于最新的先进模型，并揭示了深度神经网络中周期性激活函数在建模功能性神经影像数据方面的应用潜力。