User modeling, which learns to represent users into a low-dimensional representation space based on their past behaviors, got a surge of interest from the industry for providing personalized services to users. Previous efforts in user modeling mainly focus on learning a task-specific user representation that is designed for a single task. However, since learning task-specific user representations for every task is infeasible, recent studies introduce the concept of universal user representation, which is a more generalized representation of a user that is relevant to a variety of tasks. Despite their effectiveness, existing approaches for learning universal user representations are impractical in real-world applications due to the data requirement, catastrophic forgetting and the limited learning capability for continually added tasks. In this paper, we propose a novel continual user representation learning method, called TERACON, whose learning capability is not limited as the number of learned tasks increases while capturing the relationship between the tasks. The main idea is to introduce an embedding for each task, i.e., task embedding, which is utilized to generate task-specific soft masks that not only allow the entire model parameters to be updated until the end of training sequence, but also facilitate the relationship between the tasks to be captured. Moreover, we introduce a novel knowledge retention module with pseudo-labeling strategy that successfully alleviates the long-standing problem of continual learning, i.e., catastrophic forgetting. Extensive experiments on public and proprietary real-world datasets demonstrate the superiority and practicality of TERACON. Our code is available at https://github.com/Sein-Kim/TERACON.

翻译：用户建模通过学习将用户基于其历史行为映射到低维表示空间，为提供个性化服务而引起了工业界的广泛关注。以往的用户建模研究主要集中于为单个任务学习任务特定的用户表示。然而，由于为每个任务学习任务特定用户表示不可行，近期研究引入了通用用户表示的概念，即一种与多种任务相关的更泛化的用户表示。尽管现有通用用户表示学习方法效果显著，但受限于数据需求、灾难性遗忘以及持续添加任务时的有限学习能力，这些方法在实际应用中并不可行。本文提出一种新颖的持续用户表示学习方法TERACON，其学习能力不会随已学习任务数量的增加而受限，同时能够捕捉任务间的关系。核心思想是为每个任务引入嵌入表示（即任务嵌入），用于生成任务特定的软掩码，这不仅允许整个模型参数在训练序列结束前持续更新，还能有效捕捉任务间关系。此外，我们引入了一种基于伪标签策略的新型知识保留模块，成功缓解了持续学习中长期存在的灾难性遗忘问题。在公开和专有真实世界数据集上的大量实验证明了TERACON的优越性和实用性。我们的代码已开源至https://github.com/Sein-Kim/TERACON。