In this paper, we propose a novel text promptable surgical instrument segmentation approach to overcome challenges associated with diversity and differentiation of surgical instruments in minimally invasive surgeries. We redefine the task as text promptable, thereby enabling a more nuanced comprehension of surgical instruments and adaptability to new instrument types. Inspired by recent advancements in vision-language models, we leverage pretrained image and text encoders as our model backbone and design a text promptable mask decoder consisting of attention- and convolution-based prompting schemes for surgical instrument segmentation prediction. Our model leverages multiple text prompts for each surgical instrument through a new mixture of prompts mechanism, resulting in enhanced segmentation performance. Additionally, we introduce a hard instrument area reinforcement module to improve image feature comprehension and segmentation precision. Extensive experiments on several surgical instrument segmentation datasets demonstrate our model's superior performance and promising generalization capability. To our knowledge, this is the first implementation of a promptable approach to surgical instrument segmentation, offering significant potential for practical application in the field of robotic-assisted surgery.

翻译：本文提出了一种新颖的可文本提示手术器械分割方法，以解决微创手术中器械多样性与差异性带来的挑战。我们将该任务重新定义为可文本提示分割，从而实现对手术器械更精细的理解，并增强对新器械类型的适应能力。受视觉语言模型最新进展的启发，我们采用预训练的图像和文本编码器作为模型主干，并设计了一个基于注意力机制与卷积提示方案的可文本提示掩码解码器，用于手术器械分割预测。通过引入新的提示混合机制，模型为每种手术器械提供多个文本提示，显著提升了分割性能。此外，我们提出硬器械区域强化模块，以改善图像特征理解与分割精度。在多个手术器械分割数据集上的大量实验表明，本模型具有卓越的性能与良好的泛化能力。据我们所知，这是首个将提示式方法应用于手术器械分割的实现，为机器人辅助手术领域的实际应用提供了重要潜力。