Generalist embodied agents must perform interactive, causally-dependent reasoning, continually interacting with the environment, acquiring information, and updating plans to solve long-horizon tasks before they could be adopted in real-life scenarios. For instance, retrieving an apple from a cabinet may require opening multiple doors and drawers before the apple becomes visible and reachable, demanding sequential interaction under partial observability. However, existing benchmarks fail to systematically evaluate this essential capability. We introduce COIN, a benchmark designed to assess interactive reasoning in realistic robotic manipulation through three key contributions. First, we construct COIN-50: 50 interactive tasks in daily scenarios, and create COIN-Primitive required by causally-dependent tasks, and COIN-Composition with mid-term complexity for skill learning and generalization evaluation. Second, we develop a low-cost mobile AR teleoperation system and collect the COIN-Primitive Dataset with 50 demonstrations per primitive task (1,000 in total). Third, we develop systematic evaluation metrics about execution stability and generalization robustness to evaluate CodeAsPolicy, VLA, and language-conditioned H-VLA approaches. Our comprehensive evaluation reveals critical limitations in current methods: models struggle with interactive reasoning tasks due to significant gaps between visual understanding and motor execution. We provide fine-grained analysis of these limitations.

翻译：通用型具身智能体必须具备交互式的因果依赖推理能力，即持续与环境互动、获取信息并更新计划，以解决长时序任务，方能应用于现实场景。例如，从柜子中取出苹果可能需要先打开多个门和抽屉，待苹果可见且可触及后方可执行，这要求在部分可观测条件下进行顺序交互。然而，现有基准测试未能系统评估这一关键能力。我们提出COIN基准，通过三项核心贡献评估真实机器人操作中的交互推理能力：第一，构建COIN-50数据集——包含50个日常场景交互任务，同时创建因果依赖任务所需的COIN-Primitive子集，以及用于技能学习与泛化评估的中期复杂度COIN-Composition子集；第二，开发低成本移动AR遥操作采集系统，为每个基础任务收集50组演示数据（共1000组），形成COIN-Primitive数据集；第三，建立关于执行稳定性与泛化鲁棒性的系统评估指标，对CodeAsPolicy、VLA及语言条件化的H-VLA方法进行评测。全面评估揭示了当前方法的根本缺陷：由于视觉理解与运动执行之间存在显著差距，模型在交互推理任务中表现不佳。我们对此进行了细粒度分析。