Adapting pre-trained vision models using parameter-efficient fine-tuning (PEFT) remains challenging, as it aims to achieve performance comparable to full fine-tuning using a minimal number of trainable parameters. When applied to complex dense prediction tasks, existing methods exhibit limitations, including input-agnostic modeling and redundant cross-layer representations. To this end, we propose AdaRoute, a new adapter-style method featuring a simple mixture-of-experts (MoE) architecture. Specifically, we introduce shared expert centers, where each expert is a trainable parameter matrix. During a feedforward pass, each AdaRoute module in the network dynamically generates weight matrices tailored for the current module via a simple dynamic parameter routing mechanism, which selectively aggregates parameter matrices in the corresponding expert center. Dynamic weight matrices in AdaRoute modules facilitate low-rank adaptation in an input-dependent manner, thus generating more customized and powerful feature representations. Moreover, since AdaRoute modules across multiple network layers share the same expert center, they improve feature diversity by promoting implicit cross-layer feature interaction. Extensive experiments demonstrate the superiority of AdaRoute on diverse vision tasks, including semantic segmentation, object detection and instance segmentation, and panoptic segmentation. Code will be available at: https://bit.ly/3NZcr0H.

翻译：使用参数高效微调（PEFT）来适配预训练的视觉模型仍然具有挑战性，因为其目标是以最少的可训练参数实现与全参数微调相当的性能。当应用于复杂的密集预测任务时，现有方法表现出局限性，包括输入无关的建模和冗余的跨层表示。为此，我们提出了AdaRoute，一种采用简单混合专家（MoE）架构的新型适配器方法。具体而言，我们引入了共享专家中心，其中每个专家是一个可训练的参数量。在前向传播过程中，网络中的每个AdaRoute模块通过一个简单的动态参数路由机制，动态生成针对当前模块定制的权重矩阵，该机制选择性地聚合对应专家中心中的参数量。AdaRoute模块中的动态权重矩阵以输入依赖的方式促进低秩适配，从而生成更定制化且更强大的特征表示。此外，由于跨多个网络层的AdaRoute模块共享同一个专家中心，它们通过促进隐式的跨层特征交互，提高了特征的多样性。大量实验证明了AdaRoute在多种视觉任务上的优越性，包括语义分割、目标检测与实例分割以及全景分割。代码将在以下地址提供：https://bit.ly/3NZcr0H。