For a robot to repair its own error, it must first know it has made a mistake. One way that people detect errors is from the implicit reactions from bystanders -- their confusion, smirks, or giggles clue us in that something unexpected occurred. To enable robots to detect and act on bystander responses to task failures, we developed a novel method to elicit bystander responses to human and robot errors. Using 46 different stimulus videos featuring a variety of human and machine task failures, we collected a total of 2452 webcam videos of human reactions from 54 participants. To test the viability of the collected data, we used the bystander reaction dataset as input to a deep-learning model, BADNet, to predict failure occurrence. We tested different data labeling methods and learned how they affect model performance, achieving precisions above 90%. We discuss strategies to model bystander reactions and predict failure and how this approach can be used in real-world robotic deployments to detect errors and improve robot performance. As part of this work, we also contribute with the "Bystander Affect Detection" (BAD) dataset of bystander reactions, supporting the development of better prediction models.

翻译：为使机器人能够修复自身错误，其首先必须意识到错误的发生。人类察觉错误的方式之一，是通过旁观者的隐式反应——他们的困惑、窃笑或咯咯笑声，为我们提供了异常事件发生的线索。为赋予机器人检测并响应旁观者对任务失败的反应能力，我们开发了一种新颖的方法来引发旁观者对人类与机器人错误的反应。利用46段展示多种人机任务失败的刺激视频，我们从54名参与者处共采集了2452段网络摄像头录制的旁观者反应视频。为验证所采集数据的可行性，我们将旁观者反应数据集作为深度学习模型BADNet的输入，用于预测故障发生。我们测试了不同的数据标注方法，并探究了其对模型性能的影响，实现了超过90%的精确率。本文讨论了建模旁观者反应及预测故障的策略，并阐述了该方法如何在真实机器人部署中用于检测错误、提升机器人性能。作为本研究的一部分，我们还贡献了“旁观者情感检测”（BAD）旁观者反应数据集，以支持更优预测模型的开发。