Previous soft tissue manipulation studies assumed that the grasping point was known and the target deformation can be achieved. During the operation, the constraints are supposed to be constant, and there is no obstacles around the soft tissue. To go beyond these assumptions, a deep reinforcement learning framework with prior knowledge is proposed for soft tissue manipulation under unknown constraints, such as the force applied by fascia. The prior knowledge is represented through an intuitive manipulation strategy. As an action of the agent, a regulator factor is used to coordinate the intuitive approach and the deliberate network. A reward function is designed to balance the exploration and exploitation for large deformation. Successful simulation results verify that the proposed framework can manipulate the soft tissue while avoiding obstacles and adding new position constraints. Compared with the soft actor-critic (SAC) algorithm, the proposed framework can accelerate the training procedure and improve the generalization.

翻译：先前的软组织操作研究假设抓取点已知且目标变形可实现，操作过程中约束条件恒定，且软组织周围无障碍物。为突破这些假设，提出一种融合先验知识的深度强化学习框架，用于在未知约束（如筋膜施加的力）下进行软组织操作。先验知识通过直观操作策略进行表征。作为智能体的动作，引入调节因子协调直观方法与刻意网络。设计奖励函数以平衡大变形场景下的探索与利用。仿真结果成功验证了该框架在避障及新增位置约束条件下操纵软组织的能力。相较于软演员-评论家（SAC）算法，所提框架能加速训练过程并提升泛化性能。