The forecasting of disease progression from images is a holy grail for clinical decision making. However, this task is complicated by the inherent high dimensionality, temporal sparsity and sampling irregularity in longitudinal image acquisitions. Existing methods often rely on extracting hand-crafted features and performing time-series analysis in this vector space, leading to a loss of rich spatial information within the images. To overcome these challenges, we introduce ImageFlowNet, a novel framework that learns latent-space flow fields that evolve multiscale representations in joint embedding spaces using neural ODEs and SDEs to model disease progression in the image domain. Notably, ImageFlowNet learns multiscale joint representation spaces by combining cohorts of patients together so that information can be transferred between the patient samples. The dynamics then provide plausible trajectories of progression, with the SDE providing alternative trajectories from the same starting point. We provide theoretical insights that support our formulation of ODEs, and motivate our regularizations involving high-level visual features, latent space organization, and trajectory smoothness. We then demonstrate ImageFlowNet's effectiveness through empirical evaluations on three longitudinal medical image datasets depicting progression in retinal geographic atrophy, multiple sclerosis, and glioblastoma.

翻译：从图像预测疾病进展是临床决策的终极目标。然而，纵向图像采集固有的高维度、时间稀疏性和采样不规则性使该任务变得复杂。现有方法通常依赖于提取手工特征并在该向量空间中进行时间序列分析，导致图像内丰富空间信息的丢失。为克服这些挑战，我们提出了ImageFlowNet，这是一个新颖的框架，它利用神经ODE和SDE学习潜在空间流场，在联合嵌入空间中演化多尺度表示，以在图像域中建模疾病进展。值得注意的是，ImageFlowNet通过将患者队列组合在一起来学习多尺度联合表示空间，从而实现患者样本间的信息传递。随后，动力学模型提供了合理的进展轨迹，其中SDE能够从同一初始点生成替代轨迹。我们提供了支持ODE公式化的理论见解，并论证了涉及高级视觉特征、潜在空间组织和轨迹平滑性的正则化动机。最后，我们通过在三个描绘视网膜地理萎缩、多发性硬化症和胶质母细胞瘤进展的纵向医学图像数据集上的实证评估，证明了ImageFlowNet的有效性。