Objective.Visual-Brain Machine Interface(V-BMI) has provide a novel interaction technique for Augmented Reality (AR) industries. Several state-of-arts work has demonstates its high accuracy and real-time interaction capbilities. However, most of the studies employ EEGs devices that are rigid and difficult to apply in real-life AR glasseses application sceniraros. Here we develop a consumer-tier Visual-Brain Machine Inteface(V-BMI) system specialized for Augmented Reality(AR) glasses interactions. Approach. The developed system consists of a wearable hardware which takes advantages of fast set-up, reliable recording and comfortable wearable experience that specificized for AR glasses applications. Complementing this hardware, we have devised a software framework that facilitates real-time interactions within the system while accommodating a modular configuration to enhance scalability. Main results. The developed hardware is only 110g and 120x85x23 mm, which with 1 Tohm and peak to peak voltage is less than 1.5 uV, and a V-BMI based angry bird game and an Internet of Thing (IoT) AR applications are deisgned, we demonstrated such technology merits of intuitive experience and efficiency interaction. The real-time interaction accuracy is between 85 and 96 percentages in a commercial AR glasses (DTI is 2.24s and ITR 65 bits-min ). Significance. Our study indicates the developed system can provide an essential hardware-software framework for consumer based V-BMI AR glasses. Also, we derive several pivotal design factors for a consumer-grade V-BMI-based AR system: 1) Dynamic adaptation of stimulation patterns-classification methods via computer vision algorithms is necessary for AR glasses applications; and 2) Algorithmic localization to foster system stability and latency reduction.
翻译:摘要:目的。视觉脑机接口(V-BMI)为增强现实(AR)产业提供了新型交互技术。多项前沿研究已证明其具有高精度和实时交互能力。然而,现有研究大多采用刚性脑电设备,难以实际应用于真实场景的AR眼镜交互。为此,我们开发了一套专用于AR眼镜交互的消费级视觉脑机接口(V-BMI)系统。方法。所开发系统包括可穿戴硬件设备,该设备具备快速部署、可靠记录和舒适佩戴体验等特性,专门适配AR眼镜应用场景。配合该硬件,我们设计了一套软件框架,支持系统内实时交互,并采用模块化配置以增强可扩展性。主要结果。开发硬件仅重110克,尺寸为120×85×23毫米,输入阻抗达1 TΩ,峰峰值电压低于1.5 μV。我们设计了一款基于V-BMI的愤怒小鸟游戏和物联网AR应用,验证了该技术在直观体验与高效交互方面的优势。在商业AR眼镜上实现了85%至96%的实时交互准确率(DTI为2.24秒,ITR达65比特/分钟)。意义。研究表明,所开发系统为消费级V-BMI AR眼镜提供了关键软硬件框架。同时,我们提炼出消费级V-BMI AR系统的若干关键设计要素:1)需通过计算机视觉算法动态适配刺激模式-分类方法以适应AR眼镜应用;2)算法本地化部署有助于提升系统稳定性并降低延迟。