Indirect simultaneous positioning (ISP), where internal tissue points are placed at desired locations indirectly through the manipulation of boundary points, is a type of subtask frequently performed in robotic surgeries. Although challenging due to complex tissue dynamics, automating the task can potentially reduce the workload of surgeons. This paper presents a sim-to-real framework for learning to automate the task without interacting with a real environment, and for planning preoperatively to find the grasping points that minimize local tissue deformation. A control policy is learned using deep reinforcement learning (DRL) in the FEM-based simulation environment and transferred to real-world situation. Grasping points are planned in the simulator by utilizing the trained policy using Bayesian optimization (BO). Inconsistent simulation performance is overcome by formulating the problem as a state augmented Markov decision process (MDP). Experimental results show that the learned policy places the internal tissue points accurately, and that the planned grasping points yield small tissue deformation among the trials. The proposed learning and planning scheme is able to automate internal tissue point manipulation in surgeries and has the potential to be generalized to complex surgical scenarios.

翻译：间接同步定位（ISP）是一种通过操作边界点间接将内部组织点放置到目标位置的机器人手术子任务。尽管由于复杂的组织动力学特性而具有挑战性，但实现该任务的自动化有望减轻外科医生的负担。本文提出了一种无需真实环境交互即可完成该任务自动化的仿真到现实框架，并实现了术前自主规划以寻找能最小化局部组织变形的抓取点。在基于有限元法（FEM）的仿真环境中，通过深度强化学习（DRL）学习控制策略，并将其迁移至真实场景。抓取点通过贝叶斯优化（BO）利用训练好的策略在仿真器中规划得到。通过将问题建模为状态增强型马尔可夫决策过程（MDP），克服了仿真性能不一致的问题。实验结果表明，学习到的策略能够精准定位内部组织点，且规划出的抓取点在多次试验中产生的组织变形最小。所提出的学习与规划方案可实现手术中内部组织点的自动化操作，并具备推广至复杂手术场景的潜力。