The rapid development of generative technology opens up possibility for higher level of automation, and artificial intelligence (AI) embodiment in robotic systems is imminent. However, due to the blackbox nature of the generative technology, the generation of the knowledge and workflow scheme is uncontrolled, especially in a dynamic environment and a complex scene. This poses challenges to regulations in safety-demanding applications such as medical scenes. We argue that the unregulated generative processes from AI is fitted for low level end tasks, but intervention in the form of manual or automated regulation should happen post-workflow-generation and pre-robotic-execution. To address this, we propose a roadmap that can lead to fully automated and regulated robotic systems. In this paradigm, the high level policies are generated as structured graph data, enabling regulatory oversight and reusability, while the code base for lower level tasks is generated by generative models. Our approach aims the transitioning from expert knowledge to regulated action, akin to the iterative processes of study, practice, scrutiny, and execution in human tasks. We identify the generative and deterministic processes in a design cycle, where generative processes serve as a text-based world simulator and the deterministic processes generate the executable system. We propose State Machine Seralization Language (SMSL) to be the conversion point between text simulator and executable workflow control. From there, we analyze the modules involved based on the current literature, and discuss human in the loop. As a roadmap, this work identifies the current possible implementation and future work. This work does not provide an implemented system but envisions to inspire the researchers working on the direction in the roadmap. We implement the SMSL and D-SFO paradigm that serve as the starting point of the roadmap.
翻译:生成技术的快速发展为实现更高水平的自动化开辟了可能性,人工智能在机器人系统中的具身化已近在眼前。然而,由于生成技术的黑箱特性,知识和工作流方案的生成过程缺乏控制,尤其在动态环境和复杂场景中。这对医疗等安全敏感应用领域的规范化提出了挑战。我们认为,不受规范的人工智能生成过程适用于低层级终端任务,但应在工作流生成之后、机器人执行之前,以人工或自动化规范的形式进行干预。为此,我们提出了一条通往全自动化且规范化机器人系统的路线图。在该范式中,高层策略被生成为结构化图数据,从而支持监管审查和可重用性;而低层级任务的代码库则由生成模型生成。我们的方法旨在实现从专家知识到规范化行动的转化,类似于人类任务中学习、实践、审查与执行的迭代过程。我们识别了设计周期中的生成过程与确定性过程:生成过程充当基于文本的世界模拟器,确定性过程则生成可执行系统。我们提出状态机序列化语言(SMSL)作为文本模拟器与可执行工作流控制之间的转换点。在此基础上,基于现有文献分析了相关模块,并探讨了人在回路中的角色。作为路线图,本文指出了当前可行的实现方案和未来工作方向。本文未提供完整的实现系统,但旨在激励研究者沿该路线图方向开展工作。我们实现了SMSL与D-SFO范式,作为路线的起点。