Medical generative models, acknowledged for their high-quality sample generation ability, have accelerated the fast growth of medical applications. However, recent works concentrate on separate medical generation models for distinct medical tasks and are restricted to inadequate medical multi-modal knowledge, constraining medical comprehensive diagnosis. In this paper, we propose MedM2G, a Medical Multi-Modal Generative framework, with the key innovation to align, extract, and generate medical multi-modal within a unified model. Extending beyond single or two medical modalities, we efficiently align medical multi-modal through the central alignment approach in the unified space. Significantly, our framework extracts valuable clinical knowledge by preserving the medical visual invariant of each imaging modal, thereby enhancing specific medical information for multi-modal generation. By conditioning the adaptive cross-guided parameters into the multi-flow diffusion framework, our model promotes flexible interactions among medical multi-modal for generation. MedM2G is the first medical generative model that unifies medical generation tasks of text-to-image, image-to-text, and unified generation of medical modalities (CT, MRI, X-ray). It performs 5 medical generation tasks across 10 datasets, consistently outperforming various state-of-the-art works.

翻译：医学生成模型以其高质量样本生成能力著称，加速了医学应用的快速发展。然而，近期研究集中于针对不同医学任务的独立生成模型，受限于不充分的医学多模态知识，约束了医学综合诊断。本文提出MedM2G——医学多模态生成框架，其核心创新在于通过统一模型实现医学多模态的对齐、提取与生成。我们突破单一或两种医学模态的限制，通过统一空间中的中心对齐方法高效对齐医学多模态。特别地，我们的框架通过保留每种成像模态的医学视觉不变性来提取有价值的临床知识，从而增强多模态生成所需的特定医学信息。通过将自适应跨引导参数引入多流扩散框架，我们的模型促进了医学多模态间用于生成的灵活交互。MedM2G是首个统一文本到图像、图像到文本生成任务以及医学模态（CT、MRI、X射线）统一生成的医学生成模型。它在10个数据集上执行5项医学生成任务，持续优于各类最新方法。