In this work we present DREAM, an fMRI-to-image method for reconstructing viewed images from brain activities, grounded on fundamental knowledge of the human visual system. We craft reverse pathways that emulate the hierarchical and parallel nature of how humans perceive the visual world. These tailored pathways are specialized to decipher semantics, color, and depth cues from fMRI data, mirroring the forward pathways from visual stimuli to fMRI recordings. To do so, two components mimic the inverse processes within the human visual system: the Reverse Visual Association Cortex (R-VAC) which reverses pathways of this brain region, extracting semantics from fMRI data; the Reverse Parallel PKM (R-PKM) component simultaneously predicting color and depth from fMRI signals. The experiments indicate that our method outperforms the current state-of-the-art models in terms of the consistency of appearance, structure, and semantics. Code will be made publicly available to facilitate further research in this field.

翻译：本文提出DREAM，一种从大脑活动中重建所观看图像的fMRI-图像方法，其核心基础在于对人类视觉系统的根本性认知。我们构建了模拟人类感知视觉世界层级化与并行化特性的逆向通路。这些定制化通路专门用于从fMRI数据中解码语义、颜色和深度线索，是对视觉刺激到fMRI记录之正向通路的逆向映射。为此，两个组件模拟人视觉系统内部的逆向过程：逆向视觉关联皮层（R-VAC）组件逆转该脑区的信号通路，从fMRI数据中提取语义信息；逆向并行PKM（R-PKM）组件同步从fMRI信号中预测颜色与深度。实验表明，在表观一致性、结构一致性及语义一致性方面，本方法均优于当前最先进的模型。相关代码将公开发布，以促进该领域的进一步研究。