Cross-embodiment robot learning requires a unified action representation with consistent semantics across robot platforms. Existing representations suffer from platform-specific inconsistencies, while current solutions either maintain embodiment-specific action heads or learn latent action spaces, without fundamentally resolving the mismatch. We propose to unify robot actions in the camera frame using camera extrinsics, so that actions share consistent geometric semantics across different robot embodiments, including both single-arm and bimanual robots. However, most existing datasets lack camera extrinsic annotations, and existing offline calibration methods either suffer from local minima or require robot-specific training data. To address this gap, we present CalibAll, a training-free, robot-independent annotation pipeline that estimates camera extrinsics for offline datasets and converts heterogeneous robot actions into standardized camera-frame actions. CalibAll follows a coarse-to-fine calibration strategy: temporal PnP provides a stable initialization, followed by differentiable rendering-based refinement for high precision. Beyond extrinsics, CalibAll produces standardized TCP-pose actions and auxiliary annotations. We apply CalibAll to 16 datasets across 4 robot platforms, producing approximately 97K calibrated data episodes. Downstream simulation and real-robot experiments show that cross-embodiment pretraining with camera-frame actions achieves state-of-the-art performance.

翻译：跨实体机器人学习需要一种跨机器人平台具有一致语义的统一动作表征。现有表征存在平台特异性不一致的问题，而当前解决方案要么保留实体特定的动作头，要么学习潜在动作空间，均未能从根本上解决不匹配问题。我们提出利用相机外参将机器人动作统一至相机坐标系，使动作在不同机器人实体（包括单臂和双臂机器人）间共享一致的几何语义。然而，大多数现有数据集缺乏相机外参标注，且现有离线标定方法要么陷入局部最优，要么需要机器人特定的训练数据。为解决这一差距，我们提出CalibAll——一种无需训练、与机器人无关的标注管线，可为离线数据集估计相机外参，并将异构机器人动作转换为标准化的相机坐标系动作。CalibAll遵循由粗到精的标定策略：时序PnP提供稳定初始化，随后基于可微渲染的精化步骤实现高精度。除外参外，CalibAll还生成标准化的TCP位姿动作和辅助标注。我们将CalibAll应用于4个机器人平台的16个数据集，生成约9.7万个标定后的数据片段。下游仿真与真实机器人实验表明，基于相机坐标系动作的跨实体预训练实现了当前最优性能。