Current self-training methods such as standard self-training, co-training, tri-training, and others often focus on improving model performance on a single task, utilizing differences in input features, model architectures, and training processes. However, many tasks in natural language processing are about different but related aspects of language, and models trained for one task can be great teachers for other related tasks. In this work, we propose friend-training, a cross-task self-training framework, where models trained to do different tasks are used in an iterative training, pseudo-labeling, and retraining process to help each other for better selection of pseudo-labels. With two dialogue understanding tasks, conversational semantic role labeling and dialogue rewriting, chosen for a case study, we show that the models trained with the friend-training framework achieve the best performance compared to strong baselines.
翻译:当前的自训练方法,如标准自训练、协同训练、三训练等,通常专注于通过利用输入特征、模型架构和训练过程的差异来改进单一任务的模型性能。然而,自然语言处理中的许多任务涉及语言的不同但相关方面,针对某一任务训练的模型可以成为其他相关任务的优秀教师。在这项工作中,我们提出了一种跨任务自训练框架——朋友训练,其中执行不同任务的模型被用于迭代训练、伪标签生成和再训练过程,以相互帮助实现更好的伪标签选择。通过选取对话语义角色标注和对话改写这两个对话理解任务作为案例研究,我们展示了采用朋友训练框架训练的模型相比强基线方法取得了最佳性能。