Robots that interact with humans must adapt to individual users' preferences to operate effectively in human-centered environments. An intuitive and effective technique to learn non-expert users' preferences is through rankings of robot behaviors, e.g., trajectories, gestures, or voices. Existing techniques primarily focus on generating queries that optimize preference learning outcomes, such as sample efficiency or final preference estimation accuracy. However, the focus on outcome overlooks key user expectations in the process of providing these rankings, which can negatively impact users' adoption of robotic systems. This work proposes the Covariance Matrix Adaptation Evolution Strategies with Information Gain (CMA-ES-IG) algorithm. CMA-ES-IG explicitly incorporates user experience considerations into the preference learning process by suggesting perceptually distinct and informative trajectories for users to rank. We demonstrate these benefits through both simulated studies and real-robot experiments. CMA-ES-IG, compared to state-of-the-art alternatives, (1) scales more effectively to higher-dimensional preference spaces, (2) maintains computational tractability for high-dimensional problems, (3) is robust to noisy or inconsistent user feedback, and (4) is preferred by non-expert users in identifying their preferred robot behaviors. This project's code is available at github.com/interaction-lab/CMA-ES-IG

翻译：与人类交互的机器人必须适应个体用户的偏好，才能在以人为中心的环境中有效运行。学习非专业用户偏好的一种直观且有效的技术是通过对机器人行为（例如轨迹、手势或语音）进行排序。现有技术主要侧重于生成优化偏好学习结果的查询，例如样本效率或最终偏好估计准确性。然而，对结果的关注忽视了用户在提供这些排序过程中的关键期望，这可能对用户采用机器人系统产生负面影响。本研究提出了带有信息增益的协方差矩阵自适应进化策略（CMA-ES-IG）算法。CMA-ES-IG通过建议感知上不同且信息丰富的轨迹供用户排序，明确将用户体验考虑纳入偏好学习过程。我们通过模拟研究和真实机器人实验证明了这些优势。与最先进的替代方案相比，CMA-ES-IG（1）能更有效地扩展到更高维的偏好空间，（2）对高维问题保持计算可处理性，（3）对噪声或不一致的用户反馈具有鲁棒性，以及（4）在识别用户偏好的机器人行为方面更受非专业用户青睐。本项目的代码可在github.com/interaction-lab/CMA-ES-IG获取。