Deep learning has achieved significant advancements in medical image segmentation. Currently, obtaining accurate segmentation outcomes is critically reliant on large-scale datasets with high-quality annotations. However, noisy annotations are frequently encountered owing to the complex morphological structures of organs in medical images and variations among different annotators, which can substantially limit the efficacy of segmentation models. Motivated by the fact that medical imaging annotator can correct labeling errors during segmentation based on prior knowledge, we propose an end-to-end Staged Voxel-Level Deep Reinforcement Learning (SVL-DRL) framework for robust medical image segmentation under noisy annotations. This framework employs a dynamic iterative update strategy to automatically mitigate the impact of erroneous labels without requiring manual intervention. The key advancements of SVL-DRL over existing works include: i) formulating noisy annotations as a voxel-dependent problem and addressing it through a novel staged reinforcement learning framework which guarantees robust model convergence; ii) incorporating a voxel-level asynchronous advantage actor-critic (vA3C) module that conceptualizes each voxel as an autonomous agent, which allows each agent to dynamically refine its own state representation during training, thereby directly mitigating the influence of erroneous labels; iii) designing a novel action space for the agents, along with a composite reward function that strategically combines the Dice value and a spatial continuity metric to significantly boost segmentation accuracy while maintain semantic integrity. Experiments on three public medical image datasets demonstrates State-of-The-Art (SoTA) performance under various experimental settings, with an average improvement of over 3\% in both Dice and IoU scores.

翻译：深度学习在医学图像分割领域取得了显著进展。当前，获取精确分割结果高度依赖于具有高质量标注的大规模数据集。然而，由于医学图像中器官形态结构复杂且不同标注者之间存在差异，噪声标注现象普遍存在，这严重制约了分割模型的性能。受医学图像标注者能够依据先验知识在分割过程中修正标注误差的启发，本文提出一种端到端的分阶段体素级深度强化学习（SVL-DRL）框架，用于在噪声标注下实现鲁棒的医学图像分割。该框架采用动态迭代更新策略，无需人工干预即可自动减轻错误标签的影响。相较于现有工作，SVL-DRL的关键进展包括：i) 将噪声标注建模为体素依赖性问题，并通过新颖的分阶段强化学习框架予以解决，该框架能保证模型的鲁棒收敛性；ii) 引入体素级异步优势演员-评论家（vA3C）模块，将每个体素概念化为自主智能体，使各智能体在训练过程中动态优化其状态表示，从而直接削弱错误标签的影响；iii) 为智能体设计新颖的动作空间，并结合融合Dice系数与空间连续性度量的复合奖励函数，在保持语义完整性的同时显著提升分割精度。在三个公开医学图像数据集上的实验表明，该框架在不同实验设置下均达到最先进（SoTA）性能，Dice与IoU指标平均提升超过3%。