Brain-computer interface (BCI) facilitates direct communication between the human brain and external systems by utilizing brain signals, eliminating the need for conventional communication methods such as speaking, writing, or typing. Nevertheless, the continuous generation of brain signals in BCI frameworks poses challenges for efficient storage and real-time transmission. While considering the human brain as a semantic source, the meaningful information associated with cognitive activities often gets obscured by substantial noise present in acquired brain signals, resulting in abundant redundancy. In this paper, we propose a cross-modal brain-computer semantic communication paradigm, named EidetiCom, for decoding visual perception under limited-bandwidth constraint. The framework consists of three hierarchical layers, each responsible for compressing the semantic information of brain signals into representative features. These low-dimensional compact features are transmitted and converted into semantically meaningful representations at the receiver side, serving three distinct tasks for decoding visual perception: brain signal-based visual classification, brain-to-caption translation, and brain-to-image generation, in a scalable manner. Through extensive qualitative and quantitative experiments, we demonstrate that the proposed paradigm facilitates the semantic communication under low bit rate conditions ranging from 0.017 to 0.192 bits-per-sample, achieving high-quality semantic reconstruction and highlighting its potential for efficient storage and real-time communication of brain recordings in BCI applications, such as eidetic memory storage and assistive communication for patients.

翻译：脑机接口（BCI）通过利用脑信号，促进了人脑与外部系统之间的直接通信，从而无需依赖说话、书写或打字等传统通信方式。然而，脑机接口框架中脑信号的持续生成对高效存储和实时传输提出了挑战。在将人脑视为语义源时，与认知活动相关的有意义信息常常被采集到的脑信号中存在的大量噪声所掩盖，从而导致大量冗余。本文提出了一种名为EidetiCom的跨模态脑机语义通信范式，用于在有限带宽约束下解码视觉感知。该框架由三个层次组成，每一层负责将脑信号的语义信息压缩为代表性特征。这些低维紧凑特征在接收端被传输并转换为具有语义意义的表示，以可扩展的方式服务于解码视觉感知的三个不同任务：基于脑信号的视觉分类、脑信号到文本描述的翻译以及脑信号到图像的生成。通过广泛的定性和定量实验，我们证明所提出的范式能够在每样本0.017至0.192比特的低比特率条件下实现语义通信，获得高质量的语义重建，并突显了其在脑机接口应用（如记忆存储和患者辅助通信）中实现脑信号高效存储和实时通信的潜力。