We present a framework that integrates EEG-based visual and motor imagery (VI/MI) with robotic control to enable real-time, intention-driven grasping and placement. Motivated by the promise of BCI-driven robotics to enhance human-robot interaction, this system bridges neural signals with physical control by deploying offline-pretrained decoders in a zero-shot manner within an online streaming pipeline. This establishes a dual-channel intent interface that translates visual intent into robotic actions, with VI identifying objects for grasping and MI determining placement poses, enabling intuitive control over both what to grasp and where to place. The system operates solely on EEG via a cue-free imagery protocol, achieving integration and online validation. Implemented on a Base robotic platform and evaluated across diverse scenarios, including occluded targets or varying participant postures, the system achieves online decoding accuracies of 40.23% (VI) and 62.59% (MI), with an end-to-end task success rate of 20.88%. These results demonstrate that high-level visual cognition can be decoded in real time and translated into executable robot commands, bridging the gap between neural signals and physical interaction, and validating the flexibility of a purely imagery-based BCI paradigm for practical human-robot collaboration.
翻译:本文提出一种集成脑电视觉与运动想象(VI/MI)与机器人控制的框架,实现实时、意图驱动的抓取与放置操作。受脑机接口驱动机器人技术增强人机交互前景的启发,本系统通过在线流式处理管道中以零样本方式部署离线预训练解码器,将神经信号与物理控制相连接。该框架构建了双通道意图接口,将视觉意图转化为机器人动作:VI用于识别待抓取物体,MI用于确定放置位姿,从而实现对抓取对象与放置位置的双重直观控制。系统完全基于脑电信号运行,采用无提示想象协议,实现了集成化与在线验证。在Base机器人平台上实施并在多样化场景(包括目标遮挡或参与者姿态变化)中进行评估,系统在线解码准确率达到40.23%(VI)与62.59%(MI),端到端任务成功率为20.88%。这些结果表明,高级视觉认知能够被实时解码并转化为可执行的机器人指令,弥合了神经信号与物理交互之间的鸿沟,验证了纯想象式脑机接口范式在实际人机协作中的灵活性。