Heterogeneous robot teams operating in realistic settings often must accomplish complex missions requiring collaboration and adaptation to information acquired online. Because robot teams frequently operate in unstructured environments -- uncertain, open-world settings without prior maps -- subtasks must be grounded in robot capabilities and the physical world. While heterogeneous teams have typically been designed for fixed specifications, generative intelligence opens the possibility of teams that can accomplish a wide range of missions described in natural language. However, current large language model (LLM)-enabled teaming methods typically assume well-structured and known environments, limiting deployment in unstructured environments. We present SPINE-HT, a framework that addresses these limitations by grounding the reasoning abilities of LLMs in the context of a heterogeneous robot team through a three-stage process. Given language specifications describing mission goals and team capabilities, an LLM generates grounded subtasks which are validated for feasibility. Subtasks are then assigned to robots based on capabilities such as traversability or perception and refined given feedback collected during online operation. In simulation experiments with closed-loop perception and control, our framework achieves nearly twice the success rate compared to prior LLM-enabled heterogeneous teaming approaches. In real-world experiments with a Clearpath Jackal, a Clearpath Husky, a Boston Dynamics Spot, and a high-altitude UAV, our method achieves an 87\% success rate in missions requiring reasoning about robot capabilities and refining subtasks with online feedback. More information is provided at https://zacravichandran.github.io/SPINE-HT.

翻译：在现实场景中运行的异构机器人团队常需完成需要协作并能适应在线获取信息的复杂任务。由于机器人团队通常在非结构化环境（即不确定、无先验地图的开放世界场景）中运行，子任务必须基于机器人能力与物理世界进行具身化建模。传统异构团队多针对固定任务设计，而生成式智能为实现能完成自然语言描述的广泛任务的团队提供了可能。然而，当前基于大语言模型（LLM）的团队协作方法通常假设环境结构清晰且已知，限制了在非结构化环境中的部署。本文提出SPINE-HT框架，通过三阶段流程将LLM的推理能力具身化于异构机器人团队场景中，以解决上述局限。给定描述任务目标与团队能力的语言指令后，LLM生成经过可行性验证的具身子任务。随后根据机器人能力（如通过性或感知能力）分配子任务，并依据在线运行中收集的反馈进行动态优化。在包含闭环感知与控制的仿真实验中，本框架相比现有基于LLM的异构团队方法实现了近两倍的任务成功率。在真实世界实验中，使用Clearpath Jackal、Clearpath Husky、Boston Dynamics Spot及高空无人机组成的团队，在需要推理机器人能力并通过在线反馈优化子任务的任务中取得了87%的成功率。更多信息请访问：https://zacravichandran.github.io/SPINE-HT。