The advent of industrial robotics and autonomous systems endow human-robot collaboration in a massive scale. However, current industrial robots are restrained in co-working with human in close proximity due to inability of interpreting human agents' attention. Human attention study is non-trivial since it involves multiple aspects of the mind: perception, memory, problem solving, and consciousness. Human attention lapses are particularly problematic and potentially catastrophic in industrial workplace, from assembling electronics to operating machines. Attention is indeed complex and cannot be easily measured with single-modality sensors. Eye state, head pose, posture, and manifold environment stimulus could all play a part in attention lapses. To this end, we propose a pipeline to annotate multimodal dataset of human attention tracking, including eye tracking, fixation detection, third-person surveillance camera, and sound. We produce a pilot dataset containing two fully annotated phone assembly sequences in a realistic manufacturing environment. We evaluate existing fatigue and drowsiness prediction methods for attention lapse detection. Experimental results show that human attention lapses in production scenarios are more subtle and imperceptible than well-studied fatigue and drowsiness.

翻译：工业机器人与自主系统的涌现推动了大规模人机协作。然而，当前工业机器人在与人类近距离协同工作方面仍因无法解读人类操作者的注意力状态而受限。人类注意力研究涉及心智的多个层面：感知、记忆、问题解决与意识，极具挑战性。在工业工作场所，从电子元件装配到机器操作，人类注意力涣散尤其成问题，可能带来灾难性后果。注意力确实复杂，难以通过单模态传感器轻松测量。眼部状态、头部姿态、身体姿势及多样的环境刺激都可能在注意力涣散中发挥作用。为此，我们提出了一条标注人类注意力追踪多模态数据集的流程，包括眼动追踪、注视检测、第三人称监控摄像头和声音数据。我们在逼真的制造环境中生成了一个包含两段完整标注的手机装配序列的初步数据集。我们评估了现有疲劳与困倦预测方法在注意力涣散检测中的效果。实验结果表明，生产场景中的人类注意力涣散比研究较为深入的疲劳与困倦状态更为细微且难以察觉。