In this paper, we explore the impact of augmenting pre-trained Encoder-Decoder models, specifically T5, with linguistic knowledge for the prediction of a target task. In particular, we investigate whether fine-tuning a T5 model on an intermediate task that predicts structural linguistic properties of sentences modifies its performance in the target task of predicting sentence-level complexity. Our study encompasses diverse experiments conducted on Italian and English datasets, employing both monolingual and multilingual T5 models at various sizes. Results obtained for both languages and in cross-lingual configurations show that linguistically motivated intermediate fine-tuning has generally a positive impact on target task performance, especially when applied to smaller models and in scenarios with limited data availability.

翻译：在本文中，我们探讨了将语言知识融入预训练的编码器-解码器模型（特别是T5模型）以提升目标任务预测性能的影响。具体而言，我们研究了在预测句子结构语言属性的中间任务上微调T5模型，是否会改变其在预测句子级复杂性这一目标任务上的表现。本研究涵盖了对意大利语和英语数据集的多项实验，分别使用了不同规模的单语和多语T5模型。实验结果表明，无论是在单语还是跨语言配置下，以语言知识为驱动的中间微调通常对目标任务性能具有积极影响，尤其当应用于较小规模的模型以及数据量有限的情境时。