Reinforcement learning (RL) commonly assumes access to well-specified reward functions, which many practical applications do not provide. Instead, recently, more work has explored learning what to do from interacting with humans. So far, most of these approaches model humans as being (nosily) rational and, in particular, giving unbiased feedback. We argue that these models are too simplistic and that RL researchers need to develop more realistic human models to design and evaluate their algorithms. In particular, we argue that human models have to be personal, contextual, and dynamic. This paper calls for research from different disciplines to address key questions about how humans provide feedback to AIs and how we can build more robust human-in-the-loop RL systems.

翻译：强化学习(RL)通常假定获得明确指定的奖励功能,而许多实际应用并不提供这种功能。相反,最近,更多的工作探索了如何从与人类的互动中学习如何做。到目前为止,大多数这些工作都把人类作为(神秘的)理性的模型,特别是提供不带偏见的反馈。我们认为这些模型过于简单化,研究者需要开发更现实的人类模型来设计和评估其算法。特别是,我们主张人类模型必须是个人、背景和动态的。本文呼吁从不同学科进行研究,以解决人类如何向人工智能提供反馈以及我们如何建立更强大的流动人RL系统等关键问题。