Despite the current surge of interest in autonomous robotic systems, robot activity recognition within restricted indoor environments remains a formidable challenge. Conventional methods for detecting and recognizing robotic arms' activities often rely on vision-based or light detection and ranging (LiDAR) sensors, which require line-of-sight (LoS) access and may raise privacy concerns, for example, in nursing facilities. This research pioneers an innovative approach harnessing channel state information (CSI) measured from WiFi signals, subtly influenced by the activity of robotic arms. We developed an attention-based network to classify eight distinct activities performed by a Franka Emika robotic arm in different situations. Our proposed bidirectional vision transformer-concatenated (BiVTC) methodology aspires to predict robotic arm activities accurately, even when trained on activities with different velocities, all without dependency on external or internal sensors or visual aids. Considering the high dependency of CSI data to the environment, motivated us to study the problem of sniffer location selection, by systematically changing the sniffer's location and collecting different sets of data. Finally, this paper also marks the first publication of the CSI data of eight distinct robotic arm activities, collectively referred to as RoboFiSense. This initiative aims to provide a benchmark dataset and baselines to the research community, fostering advancements in the field of robotics sensing.

翻译：尽管当前自主机器人系统研究热潮涌现，但受限室内环境中的机器人活动识别仍是一项严峻挑战。传统的机械臂活动检测与识别方法通常依赖基于视觉或激光雷达（LiDAR）的传感器，这些传感器需要视距（LoS）通道，且可能引发隐私问题（例如在护理设施中）。本研究开创性地提出了一种创新方法，利用从WiFi信号中测量的信道状态信息（CSI）——该信息会微妙地受到机械臂活动影响。我们开发了一种基于注意力的网络，用于分类Franka Emika机械臂在不同场景下执行的八种不同活动。所提出的双向视觉Transformer-拼接（BiVTC）方法旨在精准预测机械臂活动，即便在训练数据包含不同速度活动的情况下，也能在不依赖外部/内部传感器或视觉辅助的前提下实现。考虑到CSI数据对环境的高度依赖性，我们通过系统改变嗅探器位置并采集不同数据集，研究了嗅探器位置选择问题。最后，本文还首次公开了八种不同机械臂活动的CSI数据集，统称为RoboFiSense。该举措旨在为研究社区提供基准数据集与基线方法，推动机器人感知领域的进步。