New lesion segmentation is essential to estimate the disease progression and therapeutic effects during multiple sclerosis (MS) clinical treatments. However, the expensive data acquisition and expert annotation restrict the feasibility of applying large-scale deep learning models. Since single-time-point samples with all-lesion labels are relatively easy to collect, exploiting them to train deep models is highly desirable to improve new lesion segmentation. Therefore, we proposed a coaction segmentation (CoactSeg) framework to exploit the heterogeneous data (i.e., new-lesion annotated two-time-point data and all-lesion annotated single-time-point data) for new MS lesion segmentation. The CoactSeg model is designed as a unified model, with the same three inputs (the baseline, follow-up, and their longitudinal brain differences) and the same three outputs (the corresponding all-lesion and new-lesion predictions), no matter which type of heterogeneous data is being used. Moreover, a simple and effective relation regularization is proposed to ensure the longitudinal relations among the three outputs to improve the model learning. Extensive experiments demonstrate that utilizing the heterogeneous data and the proposed longitudinal relation constraint can significantly improve the performance for both new-lesion and all-lesion segmentation tasks. Meanwhile, we also introduce an in-house MS-23v1 dataset, including 38 Oceania single-time-point samples with all-lesion labels. Codes and the dataset are released at https://github.com/ycwu1997/CoactSeg.

翻译：新病灶分割对于评估多发性硬化（MS）临床治疗中的疾病进展和治疗效果至关重要。然而，昂贵的数据采集和专家标注限制了大规模深度学习模型的应用可行性。由于单时间点全病灶标注样本相对容易收集，利用这些样本训练深度模型以改进新病灶分割具有重要价值。为此，我们提出协同分割（CoactSeg）框架，利用异构数据（即新病灶标注的双时间点数据与全病灶标注的单时间点数据）实现MS新病灶分割。CoactSeg模型采用统一架构设计，无论使用哪种类型的异构数据，都具有相同的三个输入（基线、随访及其纵向脑差异）和相同的三个输出（对应的全病灶与新病灶预测结果）。此外，我们提出一种简单有效的关联正则化方法，确保三个输出之间的纵向关系以优化模型学习。大量实验证明，利用异构数据及所提出的纵向关系约束能显著提升新病灶与全病灶分割任务的性能。同时，我们发布了包含38例大洋洲单时间点全病灶标注样本的内部MS-23v1数据集。代码与数据集已公开于https://github.com/ycwu1997/CoactSeg。