The aim of the study is to investigate the complex mechanisms of speech perception and ultimately decode the electrical changes in the brain accruing while listening to speech. We attempt to decode heard speech from intracranial electroencephalographic (iEEG) data using deep learning methods. The goal is to aid the advancement of brain-computer interface (BCI) technology for speech synthesis, and, hopefully, to provide an additional perspective on the cognitive processes of speech perception. This approach diverges from the conventional focus on speech production and instead chooses to investigate neural representations of perceived speech. This angle opened up a complex perspective, potentially allowing us to study more sophisticated neural patterns. Leveraging the power of deep learning models, the research aimed to establish a connection between these intricate neural activities and the corresponding speech sounds. Despite the approach not having achieved a breakthrough yet, the research sheds light on the potential of decoding neural activity during speech perception. Our current efforts can serve as a foundation, and we are optimistic about the potential of expanding and improving upon this work to move closer towards more advanced BCIs, better understanding of processes underlying perceived speech and its relation to spoken speech.

翻译：本研究旨在探究语音感知的复杂机制，并最终解码聆听语音时大脑产生的电信号变化。我们尝试利用深度学习方法，从颅内脑电图（iEEG）数据中解码听到的语音。研究目标在于推动用于语音合成的脑机接口（BCI）技术发展，并期望为语音感知的认知过程提供新的视角。本研究不同于传统研究聚焦于语音产生，转而考察感知语音的神经表征。这一视角开辟了复杂的研究维度，使研究者得以探索更精妙的神经模式。通过发挥深度学习模型的优势，本研究致力于建立这些复杂神经活动与对应语音信号之间的关联。尽管该研究方法尚未实现突破性进展，但揭示了感知语音时解码神经活动的潜力。当前的探索可为基础性工作，我们对扩展和优化此项研究的前景保持乐观，以推动更先进的脑机接口发展，深化对感知语音及其与产出语音关系的理解。