Aging societies, labor shortages and increasing wage costs call for assistance robots capable of autonomously performing a wide array of real-world tasks. Such open-ended robotic manipulation requires not only powerful knowledge representations and reasoning (KR&R) algorithms, but also methods for humans to instruct robots what tasks to perform and how to perform them. In this paper, we present a system for automatically generating executable robot control programs from human task demonstrations in virtual reality (VR). We leverage common-sense knowledge and game engine-based physics to semantically interpret human VR demonstrations, as well as an expressive and general task representation and automatic path planning and code generation, embedded into a state-of-the-art cognitive architecture. We demonstrate our approach in the context of force-sensitive fetch-and-place for a robotic shopping assistant. The source code is available at https://github.com/ease-crc/vr-program-synthesis.

翻译：随着社会老龄化、劳动力短缺及工资成本上升，亟需能够自主执行多样化现实任务的辅助机器人。这种开放式机器人操作不仅需要强大的知识表示与推理算法，还需要让人类能够指导机器人执行何种任务及其执行方式。本文提出一种系统，可从人类在虚拟现实环境中的任务演示自动生成可执行机器人控制程序。我们利用常识知识与基于游戏引擎的物理仿真，对人类虚拟现实演示进行语义解析，并结合表达性强的通用任务表示、自动路径规划与代码生成技术，将其嵌入到先进的认知架构中。该方法在面向机器人购物助手的力敏感取放任务场景下进行了验证。源代码见：https://github.com/ease-crc/vr-program-synthesis。