Pretraining a neural network on a large dataset is becoming a cornerstone in machine learning that is within the reach of only a few communities with large-resources. We aim at an ambitious goal of democratizing pretraining. Towards that goal, we train and release a single neural network that can predict high quality ImageNet parameters of other neural networks. By using predicted parameters for initialization we are able to boost training of diverse ImageNet models available in PyTorch. When transferred to other datasets, models initialized with predicted parameters also converge faster and reach competitive final performance.

翻译：在大型数据集上预训练神经网络正成为机器学习领域的基石，然而这一技术仅掌握在少数拥有丰富资源的科研群体手中。我们致力于实现预训练技术民主化的宏伟目标。为此，我们训练并开源了一个能够预测其他神经网络高质量ImageNet参数的单一神经网络。通过使用预测参数进行初始化，我们能够加速PyTorch中多种ImageNet模型的训练过程。当迁移至其他数据集时，经预测参数初始化的模型不仅收敛速度更快，最终性能也达到了极具竞争力的水平。