Precise robot manipulations require rich spatial information in imitation learning. Image-based policies model object positions from fixed cameras, which are sensitive to camera view changes. Policies utilizing 3D point clouds usually predict keyframes rather than continuous actions, posing difficulty in dynamic and contact-rich scenarios. To utilize 3D perception efficiently, we present RISE, an end-to-end baseline for real-world imitation learning, which predicts continuous actions directly from single-view point clouds. It compresses the point cloud to tokens with a sparse 3D encoder. After adding sparse positional encoding, the tokens are featurized using a transformer. Finally, the features are decoded into robot actions by a diffusion head. Trained with 50 demonstrations for each real-world task, RISE surpasses currently representative 2D and 3D policies by a large margin, showcasing significant advantages in both accuracy and efficiency. Experiments also demonstrate that RISE is more general and robust to environmental change compared with previous baselines. Project website: rise-policy.github.io.

翻译：摘要：精确的机器人操作需要模仿学习中丰富的空间信息。基于图像的策略从固定摄像头建模物体位置，容易受摄像头视角变化影响。利用三维点云的策略通常预测关键帧而非连续动作，在动态和接触密集场景中存在困难。为高效利用三维感知，我们提出RISE，一种面向真实世界模仿学习的端到端基线方法，可直接从单视角点云预测连续动作。它通过稀疏三维编码器将点云压缩为令牌，在添加稀疏位置编码后，使用Transformer对令牌进行特征化，最后通过扩散头将特征解码为机器人动作。在每项真实世界任务仅用50次示范训练的情况下，RISE大幅超越当前代表性的二维和三维策略，在精度和效率上均展现出显著优势。实验还表明，与先前基线方法相比，RISE对环境变化具有更强的泛化性和鲁棒性。项目网站：rise-policy.github.io。