Instructing a robot to complete an everyday task within our homes has been a long-standing challenge for robotics. While recent progress in language-conditioned imitation learning and offline reinforcement learning has demonstrated impressive performance across a wide range of tasks, they are typically limited to short-horizon tasks -- not reflective of those a home robot would be expected to complete. While existing architectures have the potential to learn these desired behaviours, the lack of the necessary long-horizon, multi-step datasets for real robotic systems poses a significant challenge. To this end, we present the Long-Horizon Manipulation (LHManip) dataset comprising 200 episodes, demonstrating 20 different manipulation tasks via real robot teleoperation. The tasks entail multiple sub-tasks, including grasping, pushing, stacking and throwing objects in highly cluttered environments. Each task is paired with a natural language instruction and multi-camera viewpoints for point-cloud or NeRF reconstruction. In total, the dataset comprises 176,278 observation-action pairs which form part of the Open X-Embodiment dataset. The full LHManip dataset is made publicly available \href{https://github.com/fedeceola/LHManip}{here}.

翻译：指导机器人在家庭中完成日常任务一直是机器人学的长期挑战。尽管近期语言条件模仿学习与离线强化学习的进展已在广泛任务中展现出显著性能，但这些方法通常局限于短时域任务——无法反映家庭机器人预期完成的任务复杂度。虽然现有架构具备学习这些期望行为的潜力，但真实机器人系统缺乏必要的长时域、多步骤数据集成为重大障碍。为此，我们提出长时域操作（LHManip）数据集，包含200个通过真实机器人遥操作演示的20种不同操作任务。这些任务涉及多个子任务，包括在高度杂乱环境中抓取、推拉、堆叠和投掷物体。每个任务均配有一条自然语言指令和用于点云或NeRF重建的多视角摄像头数据。该数据集总计包含176,278个观测-动作对，已纳入Open X-Embodiment数据集。完整LHManip数据集已在以下链接公开发布：\href{https://github.com/fedeceola/LHManip}{此处}。