Given the rise of large-scale training regimes, adapting pre-trained models to a wide range of downstream tasks has become a standard approach in machine learning. While large benefits in empirical performance have been observed, it is not yet well understood how robustness properties transfer from a pre-trained model to a downstream task. We prove that the robustness of a predictor on downstream tasks can be bound by the robustness of its underlying representation, irrespective of the pre-training protocol. Taken together, our results precisely characterize what is required of the representation function for reliable performance upon deployment.

翻译：鉴于大规模培训制度的兴起,使预先培训的模式适应一系列广泛的下游任务已成为机械学习的标准方法,虽然观察到了经验业绩的巨大效益,但尚不完全了解从预先培训的模式到下游任务的稳健性性质是如何从一个预先培训的模式转移到一个下游任务的。我们证明,下游任务的稳健性取决于其基本代表性的稳健性,而不论培训前议定书如何。加在一起,我们的结果准确地说明了部署后可靠业绩所需的代表职能。