We prove in this paper that optimizing wide ReLU neural networks (NNs) with at least one hidden layer using l2-regularization on the parameters enforces multi-task learning due to representation-learning - also in the limit of width to infinity. This is in contrast to multiple other results in the literature, in which idealized settings are assumed and where wide (ReLU)-NNs loose their ability to benefit from multi-task learning in the infinite width limit. We deduce the ability of multi-task learning from proving an exact quantitative macroscopic characterization of the learned NN in an appropriate function space.

翻译：在本文中,我们证明,在参数上使用l2正规化使至少一个隐蔽层的宽度ReLU神经网络优化,通过代表式学习(也是在宽度至无限度的限度内)使多任务学习得以实施,这与文献中的多种其他结果形成对照,在文献中假设了理想化环境,宽度(RELU)-NNS丧失了在无限宽度限度内从多任务学习中获益的能力。我们推断了多任务学习的能力,从证明在适当功能空间对学到的NNS进行精确的定量宏观定性中得出。