Programming a robotic is a complex task, as it demands the user to have a good command of specific programming languages and awareness of the robot's physical constraints. We propose a framework that simplifies robot deployment by allowing direct communication using natural language. It uses large language models (LLM) for prompt processing, workspace understanding, and waypoint generation. It also employs Augmented Reality (AR) to provide visual feedback of the planned outcome. We showcase the effectiveness of our framework with a simple pick-and-place task, which we implement on a real robot. Moreover, we present an early concept of expressive robot behavior and skill generation that can be used to communicate with the user and learn new skills (e.g., object grasping).

翻译：编程机器人是一项复杂的任务，要求用户掌握特定编程语言并了解机器人的物理约束。我们提出了一种框架，通过允许用户使用自然语言直接通信来简化机器人部署。该框架利用大语言模型进行提示处理、工作空间理解和航点生成，同时采用增强现实技术提供规划结果的视觉反馈。我们通过一个简单的拾放任务在真实机器人上验证了该框架的有效性。此外，我们提出了一种用于与用户交互和学习新技能（如物体抓取）的表现性机器人行为与技能生成的初期概念。