In this work we present DREAM, an fMRI-to-image method for reconstructing viewed images from brain activities, grounded on fundamental knowledge of the human visual system. We craft reverse pathways that emulate the hierarchical and parallel nature of how humans perceive the visual world. These tailored pathways are specialized to decipher semantics, color, and depth cues from fMRI data, mirroring the forward pathways from visual stimuli to fMRI recordings. To do so, two components mimic the inverse processes within the human visual system: the Reverse Visual Association Cortex (R-VAC) which reverses pathways of this brain region, extracting semantics from fMRI data; the Reverse Parallel PKM (R-PKM) component simultaneously predicting color and depth from fMRI signals. The experiments indicate that our method outperforms the current state-of-the-art models in terms of the consistency of appearance, structure, and semantics. Code will be made publicly available to facilitate further research in this field.

翻译：本文提出了一种名为DREAM的fMRI-to-Image方法，该方法基于人视觉系统的基础知识，从脑活动中重建所观察图像。我们构建了模拟人类视觉感知层次化与并行特性的逆向通路。这些定制化通路专门用于从fMRI数据中解码语义、色彩和深度信息，其原理与视觉刺激映射至fMRI记录的顺向通路形成镜像对应。具体实现中，两个组件模拟了人视觉系统内部的逆向过程：逆向视觉关联皮层（R-VAC）通过逆转该脑区的信息通路，从fMRI数据中提取语义信息；逆向并行PKM（R-PKM）组件则从fMRI信号中同步预测色彩与深度。实验表明，本方法在外观、结构和语义一致性方面均优于当前最优模型。为促进该领域的进一步研究，代码将公开提供。