We propose an image synthesis mechanism for multi-sequence prostate MR images conditioned on text, to control lesion presence and sequence, as well as to generate paired bi-parametric images conditioned on images e.g. for generating diffusion-weighted MR from T2-weighted MR for paired data, which are two challenging tasks in pathological image synthesis. Our proposed mechanism utilises and builds upon the recent stable diffusion model by proposing image-based conditioning for paired data generation. We validate our method using 2D image slices from real suspected prostate cancer patients. The realism of the synthesised images is validated by means of a blind expert evaluation for identifying real versus fake images, where a radiologist with 4 years experience reading urological MR only achieves 59.4% accuracy across all tested sequences (where chance is 50%). For the first time, we evaluate the realism of the generated pathology by blind expert identification of the presence of suspected lesions, where we find that the clinician performs similarly for both real and synthesised images, with a 2.9 percentage point difference in lesion identification accuracy between real and synthesised images, demonstrating the potentials in radiological training purposes. Furthermore, we also show that a machine learning model, trained for lesion identification, shows better performance (76.2% vs 70.4%, statistically significant improvement) when trained with real data augmented by synthesised data as opposed to training with only real images, demonstrating usefulness for model training.

翻译：我们提出了一种基于文本条件的多序列前列腺MRI图像合成机制，用于控制病变存在状态与序列类型，同时实现基于图像条件化的双参数图像配对生成（例如从T2加权MRI生成扩散加权MRI的配对数据），这是病理图像合成中的两个具有挑战性的任务。该机制基于并扩展了当前先进的稳定扩散模型，针对配对数据生成提出了图像条件化方法。我们采用疑似前列腺癌患者的真实二维图像切片验证方法有效性。通过盲法专家评估合成图像的真实性（区分真实与合成图像），拥有4年泌尿系MRI判读经验的放射科医生在所有测试序列上的准确率仅为59.4%（随机水平为50%）。本研究首次通过盲法专家识别疑似病灶存在性来评估生成病理的真实性，发现临床医生对真实图像与合成图像的判读表现相近，病灶识别准确率差异仅为2.9个百分点，显示出该方法在放射学培训中的应用潜力。此外，我们还证明：采用真实数据与合成数据联合训练的机器学习模型（用于病灶识别），其性能显著优于仅使用真实数据训练的模型（准确率76.2% vs 70.4%，具有统计学显著性差异），证实了该方法对模型训练的实用价值。