In multitask retrieval, a single retriever is trained to retrieve relevant contexts for multiple tasks. Despite its practical appeal, naive multitask retrieval lags behind task-specific retrieval in which a separate retriever is trained for each task. We show that it is possible to train a multitask retriever that outperforms task-specific retrievers by promoting task specialization. The main ingredients are: (1) a better choice of pretrained model (one that is explicitly optimized for multitasking) along with compatible prompting, and (2) a novel adaptive learning method that encourages each parameter to specialize in a particular task. The resulting multitask retriever is highly performant on the KILT benchmark. Upon analysis, we find that the model indeed learns parameters that are more task-specialized compared to naive multitasking without prompting or adaptive learning.

翻译：在多任务检索中，单个检索器被训练用于为多个任务检索相关上下文。尽管具有实际吸引力，朴素多任务检索的性能仍落后于为每个任务单独训练检索器的任务特定检索。我们证明，通过促进任务特化，可以训练出性能优于任务特定检索器的多任务检索器。其主要要素包括：(1) 选择更优的预训练模型（即针对多任务进行显式优化的模型），并结合兼容的提示方法；(2) 一种新颖的自适应学习方法，鼓励每个参数专注于特定任务。由此得到的多任务检索器在KILT基准上表现出高性能。分析表明，与未使用提示或自适应学习的朴素多任务方法相比，该模型确实学习了更符合任务特化的参数。