Visual-language models have advanced the development of universal models, yet their application in medical imaging remains constrained by specific functional requirements and the limited data. Current general-purpose models are typically designed with task-specific branches and heads, which restricts the shared feature space and the flexibility of model. To address these challenges, we have developed a decomposed-composed universal medical imaging paradigm (UniMed) that supports tasks at all levels. To this end, we first propose a decomposed decoder that can predict two types of outputs -- pixel and semantic, based on a defined input queue. Additionally, we introduce a composed decoder that unifies the input and output spaces and standardizes task annotations across different levels into a discrete token format. The coupled design of these two components enables the model to flexibly combine tasks and mutual benefits. Moreover, our joint representation learning strategy skilfully leverages large amounts of unlabeled data and unsupervised loss, achieving efficient one-stage pretraining for more robust performance. Experimental results show that UniMed achieves state-of-the-art performance on eight datasets across all three tasks and exhibits strong zero-shot and 100-shot transferability. We will release the code and trained models upon the paper's acceptance.

翻译：视觉-语言模型推动了通用模型的发展，但其在医学影像领域的应用仍受限于特定的功能需求和有限的数据。当前的通用模型通常设计有任务特定的分支和头部，这限制了共享特征空间和模型的灵活性。为解决这些挑战，我们开发了一种分解-组合的通用医学影像范式（UniMed），支持所有层级的任务。为此，我们首先提出了一种分解解码器，它能够基于定义的输入队列预测两种类型的输出——像素级和语义级。此外，我们引入了一种组合解码器，它统一了输入和输出空间，并将不同层级的任务标注标准化为离散的令牌格式。这两个组件的耦合设计使模型能够灵活地组合任务并实现相互增益。此外，我们的联合表征学习策略巧妙地利用了大量的未标记数据和无监督损失，实现了高效的单阶段预训练，从而获得更稳健的性能。实验结果表明，UniMed在涵盖所有三项任务的八个数据集上均取得了最先进的性能，并展现出强大的零样本和100样本迁移能力。我们将在论文被接受后发布代码和训练好的模型。