As human-robot interaction (HRI) systems advance, so does the difficulty of evaluating and understanding the strengths and limitations of these systems in different environments and with different users. To this end, previous methods have algorithmically generated diverse scenarios that reveal system failures in a shared control teleoperation task. However, these methods require directly evaluating generated scenarios by simulating robot policies and human actions. The computational cost of these evaluations limits their applicability in more complex domains. Thus, we propose augmenting scenario generation systems with surrogate models that predict both human and robot behaviors. In the shared control teleoperation domain and a more complex shared workspace collaboration task, we show that surrogate assisted scenario generation efficiently synthesizes diverse datasets of challenging scenarios. We demonstrate that these failures are reproducible in real-world interactions.

翻译：随着人机交互系统的进步，评估和理解这些系统在不同环境及与不同用户交互时的优势与局限性也变得更加困难。为此，先前的方法已通过算法生成了多样化场景，以揭示共享控制遥操作任务中的系统故障。然而，这些方法需要通过模拟机器人策略和人类行为来直接评估生成的场景，其计算成本限制了在更复杂领域的应用。因此，我们提出用代理模型来增强场景生成系统，这些模型能够预测人类和机器人的行为。在共享控制遥操作领域以及更复杂的共享工作空间协作任务中，我们证明了代理辅助场景生成能高效合成具有挑战性的多样化场景数据集，并验证了这些故障在现实交互中是可复现的。