Interactive Machine Learning (IML) seeks to integrate human expertise into machine learning processes. However, most existing algorithms cannot be applied to Realworld Scenarios because their state spaces and/or action spaces are limited to discrete values. Furthermore, the interaction of all existing methods is restricted to deciding between multiple proposals. We therefore propose a novel framework based on Bayesian Optimization (BO). Interactive Bayesian Optimization (IBO) enables collaboration between machine learning algorithms and humans. This framework captures user preferences and provides an interface for users to shape the strategy by hand. Additionally, we've incorporated a new acquisition function, Preference Expected Improvement (PEI), to refine the system's efficiency using a probabilistic model of the user preferences. Our approach is geared towards ensuring that machines can benefit from human expertise, aiming for a more aligned and effective learning process. In the course of this work, we applied our method to simulations and in a real world task using a Franka Panda robot to show human-robot collaboration.

翻译：交互式机器学习（IML）旨在将人类专业知识融入机器学习过程。然而，现有的大多数算法因其状态空间和/或动作空间局限于离散值，无法应用于真实场景。此外，所有现有方法的交互仅限于在多个候选方案之间进行选择。因此，我们提出了一种基于贝叶斯优化（BO）的新型框架。交互式贝叶斯优化（IBO）实现了机器学习算法与人类之间的协作。该框架捕获用户偏好，并提供界面供用户手动调整策略。此外，我们引入了一种新的采集函数——偏好期望改进（PEI），利用用户偏好的概率模型来提升系统效率。我们的方法致力于确保机器能够从人类专业知识中受益，以实现更协调、更高效的学习过程。在本研究中，我们将该方法应用于仿真实验，并在使用Franka Panda机器人的真实世界任务中展示了人机协作。