As human-robot interaction (HRI) systems advance, so does the difficulty of evaluating and understanding the strengths and limitations of these systems in different environments and with different users. To this end, previous methods have algorithmically generated diverse scenarios that reveal system failures in a shared control teleoperation task. However, these methods require directly evaluating generated scenarios by simulating robot policies and human actions. The computational cost of these evaluations limits their applicability in more complex domains. Thus, we propose augmenting scenario generation systems with surrogate models that predict both human and robot behaviors. In the shared control teleoperation domain and a more complex shared workspace collaboration task, we show that surrogate assisted scenario generation efficiently synthesizes diverse datasets of challenging scenarios. We demonstrate that these failures are reproducible in real-world interactions.

翻译：随着人机交互系统的进步，评估和理解这些系统在不同环境及不同用户条件下的优势与局限性也愈发困难。为此，先前的方法通过算法生成多样化场景，以揭示共享控制遥操作任务中的系统故障。然而，这些方法需要直接通过模拟机器人策略和人类行为来评估生成的场景。此类评估的计算成本限制了其在更复杂领域的适用性。因此，我们提出用代理模型增强场景生成系统，该模型可预测人类和机器人的行为。在共享控制遥操作领域及更复杂的共享工作空间协作任务中，我们证明代理辅助的场景生成能够高效合成多样化的挑战性场景数据集，并验证了这些故障在真实世界交互中的可复现性。