To act in the world, robots rely on a representation of salient task aspects: for example, to carry a cup of coffee, a robot must consider movement efficiency and cup orientation in its behaviour. However, if we want robots to act for and with people, their representations must not be just functional but also reflective of what humans care about, i.e. their representations must be aligned with humans'. In this survey, we pose that current reward and imitation learning approaches suffer from representation misalignment, where the robot's learned representation does not capture the human's representation. We suggest that because humans will be the ultimate evaluator of robot performance in the world, it is critical that we explicitly focus our efforts on aligning learned task representations with humans, in addition to learning the downstream task. We advocate that current representation learning approaches in robotics should be studied from the perspective of how well they accomplish the objective of representation alignment. To do so, we mathematically define the problem, identify its key desiderata, and situate current robot learning methods within this formalism. We conclude the survey by suggesting future directions for exploring open challenges.

翻译：为在现实世界中行动，机器人依赖对关键任务方面的表征：例如，搬运一杯咖啡时，机器人必须在行为中考虑运动效率和杯子的朝向。然而，若要机器人服务于人类并与人类协作，其表征不仅需具备功能性，更应反映人类所关注的价值，即表征必须与人类对齐。本综述指出，当前的奖励学习与模仿学习方法普遍存在表征错位问题——机器人学习到的表征未能捕捉人类的真实表征。我们认为，由于人类将是评估机器人实际表现的最終仲裁者，因此除了学习下游任务外，必须明确聚焦于将学习到的任务表征与人类对齐这一目标。本文倡导以表征对齐目标完成度作为标准，重新审视当前机器人领域的表征学习方法。为此，我们对该问题进行了数学化定义，明确了关键需求，并将现有机器人学习方法纳入该形式化框架中进行分析。最后，我们通过探讨待解决的开放挑战为未来研究指明方向。