This survey presents an overview of methods for learning from video (LfV) in the context of reinforcement learning (RL) and robotics. We focus on methods capable of scaling to large internet video datasets and, in the process, extracting foundational knowledge about the world's dynamics and physical human behaviour. Such methods hold great promise for developing general-purpose robots. We open with an overview of fundamental concepts relevant to the LfV-for-robotics setting. This includes a discussion of the exciting benefits LfV methods can offer (e.g., improved generalization beyond the available robot data) and commentary on key LfV challenges (e.g., missing information in video and LfV distribution shifts). Our literature review begins with an analysis of video foundation model techniques that can extract knowledge from large, heterogeneous video datasets. Next, we review methods that specifically leverage video data for robot learning. Here, we categorise work according to which RL knowledge modality (KM) benefits from the use of video data. We additionally highlight techniques for mitigating LfV challenges, including reviewing action representations that address missing action labels in video. Finally, we examine LfV datasets and benchmarks, before concluding with a discussion of challenges and opportunities in LfV. Here, we advocate for scalable foundation model approaches that can leverage the full range of internet video data, and that target the learning of the most promising RL KMs: the policy and dynamics model. Overall, we hope this survey will serve as a comprehensive reference for the emerging field of LfV, catalysing further research in the area and facilitating progress towards the development of general-purpose robots.

翻译：本综述概述了在强化学习与机器人学背景下从视频中学习的方法。我们重点关注能够扩展到大规模互联网视频数据集，并在此过程中提取关于世界动态与人类物理行为基础知识的方法。此类方法对于开发通用机器人具有巨大潜力。我们首先概述了视频学习应用于机器人领域的基础概念，包括讨论视频学习方法可带来的显著优势（例如超越现有机器人数据的泛化能力提升），以及对关键挑战的评述（例如视频中的信息缺失与视频学习分布偏移）。文献综述部分首先分析了能够从大规模异构视频数据集中提取知识的视频基础模型技术。接着，我们回顾了专门利用视频数据进行机器人学习的方法，并根据强化学习中哪些知识模态受益于视频数据对现有工作进行分类。此外，我们重点介绍了应对视频学习挑战的技术，包括针对视频中动作标签缺失问题的动作表示方法。最后，我们考察了现有的视频学习数据集与基准测试，并在总结部分探讨了该领域面临的挑战与机遇。我们主张发展可充分利用互联网视频数据规模优势的基础模型方法，并以学习最具潜力的强化学习知识模态——策略模型与动态模型——为目标。总体而言，我们希望本综述能成为这一新兴领域的综合参考文献，推动该方向的进一步研究，促进通用机器人技术的发展。