An object handover between a robot and a human is a coordinated action which is prone to failure for reasons such as miscommunication, incorrect actions and unexpected object properties. Existing works on handover failure detection and prevention focus on preventing failures due to object slip or external disturbances. However, there is a lack of datasets and evaluation methods that consider unpreventable failures caused by the human participant. To address this deficit, we present the multimodal Handover Failure Detection dataset, which consists of failures induced by the human participant, such as ignoring the robot or not releasing the object. We also present two baseline methods for handover failure detection: (i) a video classification method using 3D CNNs and (ii) a temporal action segmentation approach which jointly classifies the human action, robot action and overall outcome of the action. The results show that video is an important modality, but using force-torque data and gripper position help improve failure detection and action segmentation accuracy.

翻译：机器人与人类之间的物体交接是一种协调动作，由于沟通失误、错误动作和物体意外属性等原因容易失败。现有交接失败检测与预防的研究主要关注物体滑动或外部扰动导致的失败预防，但缺乏考虑由人类参与者引发的不可预防失败的评估数据集与方法。为解决这一问题，我们提出了多模态交接失败检测数据集，其中包含由人类参与者诱导的失败案例，例如忽视机器人或未释放物体。我们还提出了两种交接失败检测的基准方法：(i) 基于3D CNN的视频分类方法，以及(ii) 一种联合分类人类动作、机器人动作及动作整体结果的时间动作分割方法。结果表明，视频是关键模态，但利用力-力矩数据和夹爪位置信息有助于提升失败检测与动作分割的准确率。