Compact neural network offers many benefits for real-world applications. However, it is usually challenging to train the compact neural networks with small parameter sizes and low computational costs to achieve the same or better model performance compared to more complex and powerful architecture. This is particularly true for multitask learning, with different tasks competing for resources. We present a simple, efficient and effective multitask learning overparameterisation neural network design by overparameterising the model architecture in training and sharing the overparameterised model parameters more effectively across tasks, for better optimisation and generalisation. Experiments on two challenging multitask datasets (NYUv2 and COCO) demonstrate the effectiveness of the proposed method across various convolutional networks and parameter sizes.

翻译：紧凑型神经网络为实际应用提供了诸多优势。然而，与更复杂、更强大的架构相比，训练参数规模小、计算成本低的紧凑型神经网络，以实现相同或更优的模型性能通常颇具挑战性，尤其是在多任务学习中，不同任务会相互竞争资源。我们提出了一种简单、高效且有效的多任务学习过参数化神经网络设计方法，该方法通过在训练过程中对模型架构进行过参数化，并更有效地在任务间共享过参数化的模型参数，以提升优化效果和泛化能力。在具有挑战性的两个多任务数据集（NYUv2 和 COCO）上进行的实验表明，所提方法在各种卷积网络和参数规模下均具有有效性。