Despite the current surge of interest in autonomous robotic systems, robot activity recognition within restricted indoor environments remains a formidable challenge. Conventional methods for detecting and recognizing robotic arms' activities often rely on vision-based or light detection and ranging (LiDAR) sensors, which require line-of-sight (LoS) access and may raise privacy concerns, for example, in nursing facilities. This research pioneers an innovative approach harnessing channel state information (CSI) measured from WiFi signals, subtly influenced by the activity of robotic arms. We developed an attention-based network to classify eight distinct activities performed by a Franka Emika robotic arm in different situations. Our proposed bidirectional vision transformer-concatenated (BiVTC) methodology aspires to predict robotic arm activities accurately, even when trained on activities with different velocities, all without dependency on external or internal sensors or visual aids. Considering the high dependency of CSI data on the environment motivated us to study the problem of sniffer location selection, by systematically changing the sniffer's location and collecting different sets of data. Finally, this paper also marks the first publication of the CSI data of eight distinct robotic arm activities, collectively referred to as RoboFiSense. This initiative aims to provide a benchmark dataset and baselines to the research community, fostering advancements in the field of robotics sensing.

翻译：尽管当前对自主机器人系统的兴趣激增，但在受限室内环境中进行机器人活动识别仍是一项严峻挑战。传统检测与识别机器人臂活动的方法通常依赖视觉传感器或光探测与测距（LiDAR）传感器，这些方法需要视距（LoS）访问条件，且在养老机构等场景中可能引发隐私担忧。本研究开创性地利用受机器人臂活动微妙影响的WiFi信号信道状态信息（CSI）数据。我们开发了一种基于注意力机制的神经网络，用于分类Franka Emika机器人臂在不同情境下执行的八种不同活动。所提出的双向视觉变换器级联（BiVTC）方法旨在实现机器人臂活动的精确预测，即便训练数据包含不同速度的活动，且完全不依赖外部或内部传感器及视觉辅助设备。考虑到CSI数据对环境的强依赖性，我们通过系统改变嗅探器位置并采集多组数据，系统研究了嗅探器位置选择问题。最后，本文首次公开发布了涵盖八种机器人臂活动类型的CSI数据集（统称为RoboFiSense），旨在为研究社区提供基准数据集与基线方法，推动机器人感知领域的发展。