Ensembling is a popular method used to improve performance as a last resort. However, ensembling multiple models finetuned from a single pretrained model has been not very effective; this could be due to the lack of diversity among ensemble members. This paper proposes Multi-Ticket Ensemble, which finetunes different subnetworks of a single pretrained model and ensembles them. We empirically demonstrated that winning-ticket subnetworks produced more diverse predictions than dense networks, and their ensemble outperformed the standard ensemble on some tasks.

翻译：集合是一种常用的方法,用来作为最后手段改善业绩。然而,从单一的预先培训模式中微调的多种模型组合起来并不十分有效;这可能是因为共同成员之间缺乏多样性。本文提出多组组合,对单一预先培训模式的不同子网络进行微调和组合。我们从经验上证明,胜票子网络产生的预测比稠密的网络更为多样,其共性超过了某些任务的标准组合。