Typical black-box optimization approaches in robotics focus on learning from metric scores. However, that is not always possible, as not all developers have ground truth available. Learning appropriate robot behavior in human-centric contexts often requires querying users, who typically cannot provide precise metric scores. Existing approaches leverage human feedback in an attempt to model an implicit reward function; however, this reward may be difficult or impossible to effectively capture. In this work, we introduce SortCMA to optimize algorithm parameter configurations in high dimensions based on pairwise user preferences. SortCMA efficiently and robustly leverages user input to find parameter sets without directly modeling a reward. We apply this method to tuning a commercial depth sensor without ground truth, and to robot social navigation, which involves highly complex preferences over robot behavior. We show that our method succeeds in optimizing for the user's goals and perform a user study to evaluate social navigation results.

翻译：典型的机器人学黑箱优化方法侧重于从度量分数中学习。然而，这并非总是可行，因为并非所有开发者都能获得真实基准数据。在以人为中心的情境中学习适当的机器人行为通常需要询问用户，而用户通常无法提供精确的度量分数。现有方法利用人类反馈来尝试建模隐式奖励函数，但这种奖励可能难以或无法有效捕捉。在本研究中，我们引入SortCMA，用于基于成对用户偏好对高维算法参数配置进行优化。SortCMA能高效且鲁棒地利用用户输入，在不直接建模奖励的情况下找到参数集。我们将该方法应用于无真实基准数据情况下的商用深度传感器调参，以及涉及高度复杂机器人行为偏好的机器人社交导航任务。实验表明，我们的方法能成功优化以实现用户目标，并通过用户研究评估了社交导航结果。