Compared with the widely investigated homogeneous multi-robot collaboration, heterogeneous robots with different capabilities can provide a more efficient and flexible collaboration for more complex tasks. In this paper, we consider a more challenging heterogeneous ad hoc teamwork collaboration problem where an ad hoc robot joins an existing heterogeneous team for a shared goal. Specifically, the ad hoc robot collaborates with unknown teammates without prior coordination, and it is expected to generate an appropriate cooperation policy to improve the efficiency of the whole team. To solve this challenging problem, we leverage the remarkable potential of the large language model (LLM) to establish a decentralized heterogeneous ad hoc teamwork collaboration framework that focuses on generating reasonable policy for an ad hoc robot to collaborate with original heterogeneous teammates. A training-free hierarchical dynamic planner is developed using the LLM together with the newly proposed Interactive Reflection of Thoughts (IRoT) method for the ad hoc agent to adapt to different teams. We also build a benchmark testing dataset to evaluate the proposed framework in the heterogeneous ad hoc multi-agent tidying-up task. Extensive comparison and ablation experiments are conducted in the benchmark to demonstrate the effectiveness of the proposed framework. We have also employed the proposed framework in physical robots in a real-world scenario. The experimental videos can be found at https://youtu.be/wHYP5T2WIp0.

翻译：与广泛研究的同构多机器人协作相比，具备不同能力的异构机器人能够为更复杂的任务提供更高效、灵活的协作。本文研究一个更具挑战性的异构临时团队协作问题：一个临时机器人加入已有的异构团队以完成共同目标。具体而言，该临时机器人在没有预先协调的情况下与未知队友协作，并需要生成合适的合作策略以提升团队整体效率。为解决这一挑战性问题，我们利用大语言模型（LLM）的卓越潜力，建立了一个去中心化的异构临时团队协作框架，该框架专注于为临时机器人生成合理策略，使其能与原始异构队友协作。我们开发了一种免训练的分层动态规划器，该规划器结合LLM与新提出的交互式思维反思（IRoT）方法，使临时智能体能够适应不同的团队。我们还构建了基准测试数据集，在异构临时多智能体整理任务中评估所提框架。在基准测试中进行了广泛的对比与消融实验，以证明所提框架的有效性。我们还在真实场景中将所提框架应用于物理机器人。实验视频可在 https://youtu.be/wHYP5T2WIp0 查看。