Simulation-to-real is the task of training and developing machine learning models and deploying them in real settings with minimal additional training. This approach is becoming increasingly popular in fields such as robotics. However, there is often a gap between the simulated environment and the real world, and machine learning models trained in simulation may not perform as well in the real world. We propose a framework that utilizes a message-passing pipeline to minimize the information gap between simulation and reality. The message-passing pipeline is comprised of three modules: scene understanding, robot planning, and performance validation. First, the scene understanding module aims to match the scene layout between the real environment set-up and its digital twin. Then, the robot planning module solves a robotic task through trial and error in the simulation. Finally, the performance validation module varies the planning results by constantly checking the status difference of the robot and object status between the real set-up and the simulation. In the experiment, we perform a case study that requires a robot to make a cup of coffee. Results show that the robot is able to complete the task under our framework successfully. The robot follows the steps programmed into its system and utilizes its actuators to interact with the coffee machine and other tools required for the task. The results of this case study demonstrate the potential benefits of our method that drive robots for tasks that require precision and efficiency. Further research in this area could lead to the development of even more versatile and adaptable robots, opening up new possibilities for automation in various industries.

翻译：仿真到现实是指训练和开发机器学习模型，并将其部署到实际环境中且仅需最少额外训练的任务。该方法在机器人等领域正日益流行。然而，仿真环境与现实世界之间通常存在差距，在仿真环境中训练的机器学习模型在现实世界中可能表现不佳。我们提出一种框架，利用消息传递流程来最小化仿真与现实之间的信息差距。该消息传递流程由三个模块组成：场景理解、机器人规划和性能验证。首先，场景理解模块旨在匹配现实环境设置与其数字孪生体之间的场景布局。然后，机器人规划模块通过在仿真中反复试错来解决机器人任务。最后，性能验证模块通过持续检查现实设置与仿真中机器人和物体状态差异来调整规划结果。在实验中，我们进行了一项需要机器人制作咖啡的案例研究。结果表明，机器人能在我们的框架下成功完成任务。机器人按照其系统预设步骤，利用执行器与咖啡机及其他所需工具进行交互。该案例研究的结果证明了我们的方法在驱动机器人完成需要精度和效率的任务方面的潜在优势。对该领域的进一步研究可能推动开发出更通用、适应性更强的机器人，为各行业的自动化开辟新的可能性。