As the use of collaborative robots (cobots) in industrial manufacturing continues to grow, human action recognition for effective human-robot collaboration becomes increasingly important. This ability is crucial for cobots to act autonomously and assist in assembly tasks. Recently, skeleton-based approaches are often used as they tend to generalize better to different people and environments. However, when processing skeletons alone, information about the objects a human interacts with is lost. Therefore, we present a novel approach of integrating object information into skeleton-based action recognition. We enhance two state-of-the-art methods by treating object centers as further skeleton joints. Our experiments on the assembly dataset IKEA ASM show that our approach improves the performance of these state-of-the-art methods to a large extent when combining skeleton joints with objects predicted by a state-of-the-art instance segmentation model. Our research sheds light on the benefits of combining skeleton joints with object information for human action recognition in assembly tasks. We analyze the effect of the object detector on the combination for action classification and discuss the important factors that must be taken into account.

翻译：随着协作机器人在工业制造中的应用不断增长，实现有效人机协作的人体动作识别变得愈发重要。这种能力对于协作机器人在装配任务中自主行动并提供协助至关重要。近年来，基于骨架的方法因其对不同人员及环境具有更优的泛化能力而常被采用。然而，仅处理骨架数据会丢失人与物体交互的信息。为此，我们提出一种将物体信息融入基于骨架的动作识别的新方法。通过将物体中心作为附加骨架关节点，我们改进了两种现有先进方法。在装配数据集IKEA ASM上的实验表明，当将骨架关节点与通过先进实例分割模型预测的物体信息相结合时，我们的方法显著提升了这些先进方法的性能。本研究揭示了在装配任务中将骨架关节点与物体信息结合用于人体动作识别的优势，分析了物体检测器对动作分类中融合效果的影响，并讨论了需要重点考虑的关键因素。