Robotic manipulation requires precise spatial understanding to interact with objects in the real world. Point-based methods suffer from sparse sampling, leading to the loss of fine-grained semantics. Image-based methods typically feed RGB and depth into 2D backbones pre-trained on 3D auxiliary tasks, but their entangled semantics and geometry are sensitive to inherent depth noise in real-world that disrupts semantic understanding. Moreover, these methods focus on high-level geometry while overlooking low-level spatial cues essential for precise interaction. We propose SpatialActor, a disentangled framework for robust robotic manipulation that explicitly decouples semantics and geometry. The Semantic-guided Geometric Module adaptively fuses two complementary geometry from noisy depth and semantic-guided expert priors. Also, a Spatial Transformer leverages low-level spatial cues for accurate 2D-3D mapping and enables interaction among spatial features. We evaluate SpatialActor on multiple simulation and real-world scenarios across 50+ tasks. It achieves state-of-the-art performance with 87.4% on RLBench and improves by 13.9% to 19.4% under varying noisy conditions, showing strong robustness. Moreover, it significantly enhances few-shot generalization to new tasks and maintains robustness under various spatial perturbations. Project Page: https://shihao1895.github.io/SpatialActor

翻译：机器人操作需要精确的空间理解以与现实世界中的物体交互。基于点云的方法因稀疏采样而存在细粒度语义信息丢失的问题。基于图像的方法通常将RGB和深度信息输入到经过3D辅助任务预训练的二维骨干网络中，但其纠缠的语义与几何特征对现实场景中固有的深度噪声极为敏感，这会破坏语义理解。此外，这些方法侧重于高层几何特征，却忽视了精确交互所必需的低层空间线索。我们提出SpatialActor，一个用于鲁棒机器人操作的解耦框架，能够显式地分离语义与几何信息。语义引导几何模块自适应地融合来自含噪深度与语义引导专家先验的两种互补几何特征。同时，空间变换器利用低层空间线索实现精确的2D-3D映射，并促进空间特征间的交互。我们在超过50个任务的多种仿真和现实场景中对SpatialActor进行评估。该方法在RLBench上达到了87.4%的最先进性能，并在不同噪声条件下实现了13.9%至19.4%的性能提升，展现出强大的鲁棒性。此外，该方法显著增强了对新任务的少样本泛化能力，并在各种空间扰动下保持鲁棒性。项目页面：https://shihao1895.github.io/SpatialActor