Parameter Efficient Tuning (PET) has gained attention for reducing the number of parameters while maintaining performance and providing better hardware resource savings, but few studies investigate dense prediction tasks and interaction between modalities. In this paper, we do an investigation of efficient tuning problems on referring image segmentation. We propose a novel adapter called Bridger to facilitate cross-modal information exchange and inject task-specific information into the pre-trained model. We also design a lightweight decoder for image segmentation. Our approach achieves comparable or superior performance with only 1.61\% to 3.38\% backbone parameter updates, evaluated on challenging benchmarks. The code is available at \url{https://github.com/kkakkkka/ETRIS}.
翻译:参数高效微调(PET)因能在减少参数数量的同时保持性能并节省硬件资源而受到关注,但少有研究探讨其在密集预测任务及模态间交互中的应用。本文针对指代图像分割中的高效微调问题展开研究。我们提出了一种名为Bridger的新型适配器,以促进跨模态信息交换,并将任务特定信息注入预训练模型。同时,我们设计了一种轻量级图像分割解码器。在多个具有挑战性的基准测试中,我们的方法仅需更新1.61%至3.38%的骨干网络参数,即可达到与现有方法相当甚至更优的性能。代码已开源在\url{https://github.com/kkakkkka/ETRIS}。