The conventional training approaches often face challenges in balancing the breadth of multi-task learning (MTL) with the depth of single-task learning (STL). To address this issue, we introduce the Multi-Task to Single-Task (MT2ST) framework, a groundbreaking approach that can combine the generalizability of MTL with the precision of STL. Our work include two strategies: 'Diminish' and 'Switch'. 'Diminish' Strategy will gradually reduce the influence of auxiliary tasks, while the 'Switch' strategy involves a shift from multi-tasking to single-tasking at a specific timepoint at the training process. In this paper, we propose the Multi-Task to Single-Task (MT2ST) framework, a novel approach that significantly enhances the efficiency and accuracy of word embedding training while concurrently addressing prevalent issues such as overfitting. Our empirical studies demonstrate that MT2ST can reduce training time by 67% when contrasted with single-task learning approaches, and by 13% compared to traditional multi-task learning methods. These findings underscore MT2ST's potential to be a powerful tools for word embedding training acceleration. The code implementation is can be found at: https://github.com/NoakLiu/MT2ST-Word-Embeddings-Acceleration.
翻译:传统的训练方法往往难以平衡多任务学习(MTL)的广度与单任务学习(STL)的深度。为解决这一问题,我们提出了多任务到单任务(MT2ST)框架,这是一种开创性的方法,能够将MTL的泛化能力与STL的精确性相结合。我们的工作包含两种策略:“减弱”与“切换”。“减弱”策略会逐步降低辅助任务的影响,而“切换”策略则涉及在训练过程的特定时间点从多任务模式转向单任务模式。在本文中,我们提出的多任务到单任务(MT2ST)框架,作为一种新颖方法,在显著提升词嵌入训练效率和准确性的同时,也解决了诸如过拟合等普遍问题。我们的实证研究表明,与单任务学习方法相比,MT2ST能减少67%的训练时间;与传统多任务学习方法相比,则能减少13%的训练时间。这些发现凸显了MT2ST作为一种强大工具在加速词嵌入训练方面的潜力。代码实现可见于:https://github.com/NoakLiu/MT2ST-Word-Embeddings-Acceleration。