Accurate segmentation is essential for echocardiography-based assessment of cardiovascular diseases (CVDs). However, the variability among sonographers and the inherent challenges of ultrasound images hinder precise segmentation. By leveraging the joint representation of image and text modalities, Vision-Language Segmentation Models (VLSMs) can incorporate rich contextual information, potentially aiding in accurate and explainable segmentation. However, the lack of readily available data in echocardiography hampers the training of VLSMs. In this study, we explore using synthetic datasets from Semantic Diffusion Models (SDMs) to enhance VLSMs for echocardiography segmentation. We evaluate results for two popular VLSMs (CLIPSeg and CRIS) using seven different kinds of language prompts derived from several attributes, automatically extracted from echocardiography images, segmentation masks, and their metadata. Our results show improved metrics and faster convergence when pretraining VLSMs on SDM-generated synthetic images before finetuning on real images. The code, configs, and prompts are available at https://github.com/naamiinepal/synthetic-boost.

翻译：精确分割对于基于超声心动图的心血管疾病评估至关重要。然而，超声技师之间的操作差异以及超声图像固有的挑战阻碍了精确分割。通过利用图像和文本模态的联合表示，视觉-语言分割模型（VLSMs）能够融入丰富的上下文信息，可能有助于实现精准且可解释的分割。然而，超声心动图领域缺乏现成数据，导致VLSM训练受限。本研究探索利用来自语义扩散模型（SDMs）的合成数据集来增强用于超声心动图分割的VLSM。我们基于从超声心动图图像、分割掩膜及其元数据中自动提取的多种属性生成的七种不同语言提示，评估了两类主流VLSM（CLIPSeg和CRIS）的性能。结果表明，在真实图像上微调前，先使用SDM生成的合成图像预训练VLSM，可提升评估指标并加速收敛。相关代码、配置及提示已开源至https://github.com/naamiinepal/synthetic-boost。