We propose an instructions-based approach for robot programming where the programmer interacts with the robot by issuing simple commands in a scripting language, like python. Internally, these commands make use of pre-programmed motion and manipulation skills coordinated by a behaviour tree task controller. A knowledge graph keeps track of the state of the robot and the environment and of all the instructions given to the robot by the programmer. This allows to easily transform sequences of instructions into new skills that can be reused in the same or in other tasks. An advantage of this approach is that the programmer does not need to be located physically next to the robot, but can work remotely, either with a physical robot or with a digital twin. To demonstrate the concept, we show an interactive simulation of a robot manipulator in a pick and place scenario.

翻译：我们提出了一种基于指令的机器人编程方法，程序员可通过脚本语言（如Python）向机器人发送简单指令进行交互。在系统内部，这些指令利用由行为树任务控制器协调的预编程运动与操作技能。知识图谱持续跟踪机器人及环境状态，并记录程序员向机器人发出的所有指令。这使得指令序列能够轻松转化为新技能，并可在相同或其他任务中重复使用。该方法的优势在于程序员无需身处机器人物理现场，即可远程操作实体机器人或数字孪生体。为验证该概念，我们在拾取放置场景中展示了机械臂的交互式仿真系统。