Excavators are crucial for diverse tasks such as construction and mining, while autonomous excavator systems enhance safety and efficiency, address labor shortages, and improve human working conditions. Different from the existing modularized approaches, this paper introduces ExACT, an end-to-end autonomous excavator system that processes raw LiDAR, camera data, and joint positions to control excavator valves directly. Utilizing the Action Chunking with Transformers (ACT) architecture, ExACT employs imitation learning to take observations from multi-modal sensors as inputs and generate actionable sequences. In our experiment, we build a simulator based on the captured real-world data to model the relations between excavator valve states and joint velocities. With a few human-operated demonstration data trajectories, ExACT demonstrates the capability of completing different excavation tasks, including reaching, digging and dumping through imitation learning in validations with the simulator. To the best of our knowledge, ExACT represents the first instance towards building an end-to-end autonomous excavator system via imitation learning methods with a minimal set of human demonstrations. The video about this work can be accessed at https://youtu.be/NmzR_Rf-aEk.

翻译：挖掘机在建筑、采矿等多种任务中至关重要，而自主挖掘机系统能够提升安全性与效率，缓解劳动力短缺问题并改善人类工作条件。与现有模块化方法不同，本文提出ExACT——一种端到端自主挖掘机系统，可直接处理原始激光雷达、相机数据与关节位置信息来控制挖掘机阀门。基于Transformer动作分块（ACT）架构，ExACT通过模仿学习接收多模态传感器观测数据作为输入，生成可执行的动作序列。实验中我们基于采集的真实世界数据构建模拟器，以建模挖掘机阀门状态与关节速度之间的关联。借助少量人工操作演示数据轨迹，ExACT在模拟器验证中展现出通过模仿学习完成不同挖掘任务的能力，包括触达、挖掘与倾倒。据我们所知，ExACT是首个通过模仿学习方法、仅需最少人工演示即可构建端到端自主挖掘机系统的实例。本工作相关视频可访问https://youtu.be/NmzR_Rf-aEk获取。