Information retrieval systems have historically relied on explicit query formulation, requiring users to translate their information needs into text. This process is particularly disruptive during reading tasks, where users must interrupt their natural flow to formulate queries. We present DEEPER (Dense Electroencephalography Passage Retrieval), a novel framework that enables direct retrieval of relevant passages from users' neural signals during naturalistic reading without intermediate text translation. Building on dense retrieval architectures, DEEPER employs a dual-encoder approach with specialised components for processing neural data, mapping EEG signals and text passages into a shared semantic space. Through careful architecture design and cross-modal negative sampling strategies, our model learns to align neural patterns with their corresponding textual content. Experimental results on the ZuCo dataset demonstrate that direct brain-to-passage retrieval significantly outperforms current EEG-to-text baselines, achieving a 571% improvement in Precision@1. Our ablation studies reveal that the model successfully learns aligned representations between EEG and text modalities (0.29 cosine similarity), while our hard negative sampling strategy contributes to overall performance increases.
翻译:信息检索系统历来依赖于显式查询构建,要求用户将信息需求转化为文本。这一过程在阅读任务中尤为干扰,用户必须中断自然阅读流程来构建查询。本文提出DEEPER(密集脑电图段落检索)框架,该创新系统能够在自然阅读过程中直接从用户神经信号中检索相关段落,无需中间文本转换。基于密集检索架构,DEEPER采用双编码器方法,配备专门处理神经数据的组件,将脑电信号与文本段落映射到共享语义空间。通过精细的架构设计和跨模态负采样策略,我们的模型学习将神经模式与其对应文本内容对齐。在ZuCo数据集上的实验结果表明,直接脑信号到段落检索显著优于当前脑电到文本基线方法,在Precision@1指标上实现了571%的提升。消融研究表明,该模型成功学习了脑电与文本模态间的对齐表示(余弦相似度0.29),而我们的困难负采样策略对整体性能提升具有重要贡献。