Human organs constantly undergo anatomical changes due to a complex mix of short-term (e.g., heartbeat) and long-term (e.g., aging) factors. Evidently, prior knowledge of these factors will be beneficial when modeling their future state, i.e., via image generation. However, most of the medical image generation tasks only rely on the input from a single image, thus ignoring the sequential dependency even when longitudinal data is available. Sequence-aware deep generative models, where model input is a sequence of ordered and timestamped images, are still underexplored in the medical imaging domain that is featured by several unique challenges: 1) Sequences with various lengths; 2) Missing data or frame, and 3) High dimensionality. To this end, we propose a sequence-aware diffusion model (SADM) for the generation of longitudinal medical images. Recently, diffusion models have shown promising results in high-fidelity image generation. Our method extends this new technique by introducing a sequence-aware transformer as the conditional module in a diffusion model. The novel design enables learning longitudinal dependency even with missing data during training and allows autoregressive generation of a sequence of images during inference. Our extensive experiments on 3D longitudinal medical images demonstrate the effectiveness of SADM compared with baselines and alternative methods. The code is available at https://github.com/ubc-tea/SADM-Longitudinal-Medical-Image-Generation.

翻译：人体器官因短期（如心跳）与长期（如衰老）等多重复杂因素的共同作用而持续发生解剖结构变化。显然，在通过图像生成技术建模器官未来状态时，对这些因素的先验知识将有所助益。然而，现有医学图像生成任务大多仅依赖单张图像输入，即便获取了纵向数据也常忽略其序列依赖关系。在医学影像领域，序列感知深度生成模型（即输入为带时间戳的排序图像序列）仍处于未充分探索阶段，该领域面临三大独特挑战：1）序列长度不一；2）数据或图像帧缺失；3）高维度问题。为此，我们提出序列感知扩散模型（SADM）用于生成纵向医学图像。近年来，扩散模型在高保真图像生成方面展现出令人瞩目的成果。本研究通过将序列感知Transformer作为扩散模型的 conditioning 模块，对这一前沿技术进行拓展。该创新设计使模型能够在训练阶段即使存在缺失数据的情况下仍可学习纵向依赖关系，并在推理阶段实现图像序列的自回归生成。基于3D纵向医学图像的大量实验表明，SADM相较基线方法及替代方案均展现出显著有效性。代码已开源至：https://github.com/ubc-tea/SADM-Longitudinal-Medical-Image-Generation。