One of the challenges in robotics is to enable robotic units with the reasoning capability that would be robust enough to execute complex tasks in dynamic environments. Recent advances in LLMs have positioned them as go-to tools for simple reasoning tasks, motivating the pioneering work of Liang et al. [35] that uses an LLM to translate natural language commands into low-level static execution plans for robotic units. Using LLMs inside robotics systems brings their generalization to a new level, enabling zero-shot generalization to new tasks. This paper extends this prior work to dynamic environments. We propose InCoRo, a system that uses a classical robotic feedback loop composed of an LLM controller, a scene understanding unit, and a robot. Our system continuously analyzes the state of the environment and provides adapted execution commands, enabling the robot to adjust to changing environmental conditions and correcting for controller errors. Our system does not require any iterative optimization to learn to accomplish a task as it leverages in-context learning with an off-the-shelf LLM model. Through an extensive validation process involving two standardized industrial robotic units -- SCARA and DELTA types -- we contribute knowledge about these robots, not popular in the community, thereby enriching it. We highlight the generalization capabilities of our system and show that (1) in-context learning in combination with the current state-of-the-art LLMs is an effective way to implement a robotic controller; (2) in static environments, InCoRo surpasses the prior art in terms of the success rate; (3) in dynamic environments, we establish new state-of-the-art for the SCARA and DELTA units, respectively. This research paves the way towards building reliable, efficient, intelligent autonomous systems that adapt to dynamic environments.

翻译：机器人领域的一大挑战在于赋予机器人单元足够的推理能力，使其能够在动态环境中稳健执行复杂任务。大语言模型（LLMs）的最新进展使其成为简单推理任务的首选工具，这促使Liang等人[35]开创性地利用LLM将自然语言指令转化为机器人单元的低层次静态执行方案。将LLM应用于机器人系统显著提升了其泛化能力，实现了对新任务的零样本泛化。本文将此先前工作扩展至动态环境。我们提出InCoRo系统，该系统采用由LLM控制器、场景理解单元和机器人组成的经典机器人反馈回路。我们的系统持续分析环境状态并提供自适应执行指令，使机器人能够适应不断变化的环境条件并纠正控制器错误。该系统无需任何迭代优化来学习完成任务，因为它利用现成的LLM模型进行情境学习。通过涉及两种标准化工业机器人单元（SCARA型和DELTA型）的广泛验证过程，我们贡献了关于这类在社区中不受欢迎的机器人的知识，从而丰富了该领域。我们强调了系统的泛化能力，并表明：（1）情境学习与当前最先进的LLM相结合是实现机器人控制器的有效方式；（2）在静态环境中，InCoRo在成功率上超越了先前技术；（3）在动态环境中，我们分别为SCARA和DELTA单元建立了新的最先进水平。这项研究为构建能够适应动态环境的可靠、高效、智能的自主系统铺平了道路。