Autonomous robots in endovascular operations have the potential to navigate circulatory systems safely and reliably while decreasing the susceptibility to human errors. However, there are numerous challenges involved with the process of training such robots such as long training duration due to sample inefficiency of machine learning algorithms and safety issues arising from the interaction between the catheter and the endovascular phantom. Physics simulators have been used in the context of endovascular procedures, but they are typically employed for staff training and generally do not conform to the autonomous cannulation goal. Furthermore, most current simulators are closed-source which hinders the collaborative development of safe and reliable autonomous systems. In this work, we introduce CathSim, an open-source simulation environment that accelerates the development of machine learning algorithms for autonomous endovascular navigation. We first simulate the high-fidelity catheter and aorta with the state-of-the-art endovascular robot. We then provide the capability of real-time force sensing between the catheter and the aorta in the simulation environment. We validate our simulator by conducting two different catheterisation tasks within two primary arteries using two popular reinforcement learning algorithms, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC). The experimental results show that using our open-source simulator, we can successfully train the reinforcement learning agents to perform different autonomous cannulation tasks.

翻译：自主机器人在血管内手术中有望安全可靠地导航循环系统，同时减少人为错误的易感性。然而，训练此类机器人涉及诸多挑战，例如机器学习算法样本效率低导致训练周期长，以及导管与血管内模体交互引发的安全问题。物理模拟器已用于血管内手术场景，但通常用于人员培训，且一般不符合自主插管目标。此外，现有模拟器大多为闭源，阻碍了安全可靠自主系统的协同开发。本文提出CathSim——一种开源仿真环境，可加速血管内自主导航机器学习算法的开发。我们首先利用最先进的血管内机器人模拟高保真导管与主动脉，然后在仿真环境中提供导管与主动脉之间的实时力传感能力。通过使用两种主流强化学习算法——近端策略优化（PPO）和软演员-评论家（SAC）——在两条主要动脉上执行两项不同的插管任务，我们验证了模拟器的有效性。实验结果表明，利用我们的开源模拟器，可成功训练强化学习智能体完成不同的自主插管任务。