Language agents that interact with the world on their own have great potential for automating digital tasks. While large language model (LLM) agents have made progress in understanding and executing tasks such as textual games and webpage control, many real-world tasks also require collaboration with humans or other LLMs in equal roles, which involves intent understanding, task coordination, and communication. To test LLM's ability to collaborate, we design a blocks-world environment, where two agents, each having unique goals and skills, build a target structure together. To complete the goals, they can act in the world and communicate in natural language. Under this environment, we design increasingly challenging settings to evaluate different collaboration perspectives, from independent to more complex, dependent tasks. We further adopt chain-of-thought prompts that include intermediate reasoning steps to model the partner's state and identify and correct execution errors. Both human-machine and machine-machine experiments show that LLM agents have strong grounding capacities, and our approach significantly improves the evaluation metric.

翻译：能够自主与环境交互的语言代理在自动化数字任务方面具有巨大潜力。尽管大语言模型代理在理解和执行如文本游戏及网页控制等任务上已取得进展，但许多现实世界的任务还需要与人类或其他语言模型以平等角色进行协作，这涉及意图理解、任务协调与沟通。为测试语言模型的协作能力，我们设计了一个积木世界环境，其中两个代理各自拥有独特的目标和技能，共同构建一个目标结构。为达成目标，它们可以在此世界中行动并用自然语言进行交流。在该环境下，我们设计了从独立任务到更复杂、依赖任务的日益具有挑战性的设置，以评估不同的协作视角。我们进一步采用了包含中间推理步骤的思维链提示，以模拟伙伴的状态，并识别与纠正执行错误。人机及机机实验均表明，语言模型代理具有较强的理解能力，我们的方法显著提升了评估指标。