Visual feedback plays a crucial role in the process of amputation patients completing grasping in the field of prosthesis control. However, for blind and visually impaired (BVI) amputees, the loss of both visual and grasping abilities makes the "easy" reach-and-grasp task a feasible challenge. In this paper, we propose a novel multi-sensory prosthesis system helping BVI amputees with sensing, navigation and grasp operations. It combines modules of voice interaction, environmental perception, grasp guidance, collaborative control, and auditory/tactile feedback. In particular, the voice interaction module receives user instructions and invokes other functional modules according to the instructions. The environmental perception and grasp guidance module obtains environmental information through computer vision, and feedbacks the information to the user through auditory feedback modules (voice prompts and spatial sound sources) and tactile feedback modules (vibration stimulation). The prosthesis collaborative control module obtains the context information of the grasp guidance process and completes the collaborative control of grasp gestures and wrist angles of prosthesis in conjunction with the user's control intention in order to achieve stable grasp of various objects. This paper details a prototyping design (named viia-hand) and presents its preliminary experimental verification on healthy subjects completing specific reach-and-grasp tasks. Our results showed that, with the help of our new design, the subjects were able to achieve a precise reach and reliable grasp of the target objects in a relatively cluttered environment. Additionally, the system is extremely user-friendly, as users can quickly adapt to it with minimal training.

翻译：在假肢控制领域，视觉反馈对于截肢患者完成抓取动作至关重要。然而，对于盲人和视障截肢者而言，视觉与抓握能力的双重丧失使得"简单"的伸手抓取任务成为可行性的挑战。本文提出了一种新型多模态假肢系统，通过感知、导航与抓取操作协助盲视截肢者。该系统集成语音交互、环境感知、抓取引导、协同控制以及听觉/触觉反馈模块。其中，语音交互模块接收用户指令并调用其他功能模块；环境感知与抓取引导模块通过计算机视觉获取环境信息，并通过听觉反馈模块（语音提示与空间声源）及触觉反馈模块（振动刺激）将信息反馈给用户；假肢协同控制模块获取抓取引导过程的上下文信息，结合用户控制意图完成假肢抓取姿态与腕部角度的协同控制，以实现对不同物体的稳定抓取。本文详细阐述了原型设计（命名为viia-hand），并展示了在健康受试者完成特定伸手抓取任务中的初步实验验证。结果表明，借助本新型设计，受试者能在相对杂乱的环境中精确触及并可靠抓取目标物体。此外，系统具有极佳的用户友好性，用户经过极少量训练即可快速适应。