Previous soft tissue manipulation studies assumed that the grasping point was known and the target deformation can be achieved. During the operation, the constraints are supposed to be constant, and there is no obstacles around the soft tissue. To go beyond these assumptions, a deep reinforcement learning framework with prior knowledge is proposed for soft tissue manipulation under unknown constraints, such as the force applied by fascia. The prior knowledge is represented through an intuitive manipulation strategy. As an action of the agent, a regulator factor is used to coordinate the intuitive approach and the deliberate network. A reward function is designed to balance the exploration and exploitation for large deformation. Successful simulation results verify that the proposed framework can manipulate the soft tissue while avoiding obstacles and adding new position constraints. Compared with the soft actor-critic (SAC) algorithm, the proposed framework can accelerate the training procedure and improve the generalization.

翻译：先前关于软组织操作的研究假设抓取点已知且目标变形可达，操作过程中约束条件恒定且软组织周围无动障碍物。为突破上述假设，本文提出一种融合先验知识的深度强化学习框架，用于处理未知约束（如筋膜施加作用力）下的软组织操作。该框架通过直观操作策略表征先验知识，并采用调节因子协调直观方法与深思网络作为智能体的动作。针对大变形场景，设计了平衡探索与利用的奖励函数。仿真结果表明，所提框架能够在规避障碍物与新增位置约束的同时实现软组织操作。相较于软演员-评论家（SAC）算法，该框架可加速训练过程并提升泛化能力。