Vision-Language Models (VLMs) have recently demonstrated strong capabilities in mapping multimodal observations to robot behaviors. However, most current approaches rely on end-to-end visuomotor policies that remain opaque and difficult to analyze, limiting their use in real-world robotic applications. In contrast, classical robotic systems often rely on structured policy representations that provide interpretability, modularity, and reactive execution. This work investigates how foundation models can be specialized to generate structured robot policies grounded in multimodal perception, bridging high-dimensional learning and symbolic control. We propose a neuro-symbolic approach in which a VLM synthesizes executable Behavior Tree policies from visual observations, natural language instructions, and structured system specifications. To enable scalable supervision without manual annotation, we introduce an automated pipeline that generates a synthetic multimodal dataset of domain-randomized scenes paired with instruction-policy examples produced by a foundation model. By decoupling structured task decomposition under constrained symbolic grammars from hardware-specific motor control, we demonstrate that a 12B-parameter model can learn structured spatial-symbolic mappings required for executable BT synthesis, solely through in-silico supervision. Real-world physical experiments on two heterogeneous robotic manipulators confirm that these structurally constrained policies achieve zero-shot transfer to real-world environments. The results emphasize that the data bottleneck in robotic planning can be bypassed by procedurally synthesizing high-fidelity, neuro-symbolic training data.

翻译：视觉-语言模型（VLMs）近期展现出将多模态观测映射至机器人行为的强大能力。然而，现有方法大多依赖端到端视觉运动策略，其黑箱特性与难以分析的缺陷制约了在实际机器人场景中的部署。传统机器人系统则常采用结构化策略表征，具备可解释性、模块化与反应式执行的优势。本研究探索如何将基础模型专业化，使其生成基于多模态感知的结构化机器人策略，从而弥合高维学习与符号控制之间的鸿沟。我们提出神经符号方法：通过视觉观测、自然语言指令与结构化系统规范，让VLM自动合成可执行的行为树策略。为规避人工标注实现可扩展监督，我们构建自动化流水线，利用基础模型生成与指令-策略示例配对的领域随机化场景合成多模态数据集。通过将约束符号语法下的结构化任务分解与硬件相关的运动控制解耦，我们证明仅需硅基监督，一个120亿参数模型即可学习结构化空间-符号映射，实现可执行BT策略的合成。在两类异构机械臂上的真实物理实验证实，这些结构约束策略能零样本迁移至真实环境。研究结果强调，通过程序化合成高保真神经符号训练数据，可绕过机器人规划中的数据瓶颈问题。