Attention (and distraction) recognition is a key factor in improving human-robot collaboration. We present an assembly scenario where a human operator and a cobot collaborate equally to piece together a gearbox. The setup provides multiple opportunities for the cobot to adapt its behavior depending on the operator's attention, which can improve the collaboration experience and reduce psychological strain. As a first step, we recognize the areas in the workspace that the human operator is paying attention to, and consequently, detect when the operator is distracted. We propose a novel deep-learning approach to develop an attention recognition model. First, we train a convolutional neural network to estimate the gaze direction using a publicly available image dataset. Then, we use transfer learning with a small dataset to map the gaze direction onto pre-defined areas of interest. Models trained using this approach performed very well in leave-one-subject-out evaluation on the small dataset. We performed an additional validation of our models using the video snippets collected from participants working as an operator in the presented assembly scenario. Although the recall for the Distracted class was lower in this case, the models performed well in recognizing the areas the operator paid attention to. To the best of our knowledge, this is the first work that validated an attention recognition model using data from a setting that mimics industrial human-robot collaboration. Our findings highlight the need for validation of attention recognition solutions in such full-fledged, non-guided scenarios.

翻译：注意力（及分心）识别是提升人机协作的关键因素。我们提出一个装配场景，其中人类操作员与协作机器人平等协作组装齿轮箱。该设置提供了多种机会，使协作机器人能够根据操作员的注意力调整其行为，从而改善协作体验并减少心理压力。作为第一步，我们识别工作空间中人类操作员关注的区域，并检测操作员何时分心。我们提出了一种新颖的深度学习方法开发注意力识别模型。首先，我们训练了一个卷积神经网络，使用公开图像数据集估计凝视方向。然后，我们利用小规模数据集进行迁移学习，将凝视方向映射到预定义感兴趣区域。使用该方法训练的模型在小规模数据集上的留一被试评估中表现非常出色。我们进一步使用从参与上述装配场景的操作员中收集的视频片段对模型进行额外验证。尽管在此情况下"分心"类别的召回率较低，但模型在识别操作员关注的区域方面表现良好。据我们所知，这是首次使用模拟工业人机协作场景的数据验证注意力识别模型的研究。我们的发现强调了在完整、非引导式场景中验证注意力识别解决方案的必要性。