This paper presents a novel framework for real-time human action recognition in industrial contexts, using standard 2D cameras. We introduce a complete pipeline for robust and real-time estimation of human joint kinematics, input to a temporally smoothed Transformer-based network, for action recognition. We rely on a new dataset including 11 subjects performing various actions, to evaluate our approach. Unlike most of the literature that relies on joint center positions (JCP) and is offline, ours uses biomechanical prior, eg. joint angles, for fast and robust real-time recognition. Besides, joint angles make the proposed method agnostic to sensor and subject poses as well as to anthropometric differences, and ensure robustness across environments and subjects. Our proposed learning model outperforms the best baseline model, running also in real-time, along various metrics. It achieves 88% accuracy and shows great generalization ability, for subjects not facing the cameras. Finally, we demonstrate the robustness and usefulness of our technique, through an online interaction experiment, with a simulated robot controlled in real-time via the recognized actions.

翻译：本文提出了一种新颖的框架，用于在工业场景下使用标准二维摄像头进行实时人体动作识别。我们引入了一套完整的流程，用于鲁棒且实时地估计人体关节运动学参数，并将其输入到一个经过时间平滑的基于Transformer的网络中进行动作识别。我们依赖一个包含11名受试者执行多种动作的新数据集来评估我们的方法。与大多数依赖关节中心位置且为离线处理的文献不同，我们的方法利用生物力学先验知识（例如关节角度）进行快速、鲁棒的实时识别。此外，关节角度的使用使得所提方法对传感器与受试者姿态以及人体测量学差异具有无关性，并确保了在不同环境和受试者间的鲁棒性。我们提出的学习模型在多项指标上均优于同样实时运行的最佳基线模型。其识别准确率达到88%，并对未面向摄像头的受试者表现出出色的泛化能力。最后，我们通过一项在线交互实验，利用识别出的动作实时控制一个模拟机器人，展示了我们技术的鲁棒性和实用性。