User modeling, which learns to represent users into a low-dimensional representation space based on their past behaviors, got a surge of interest from the industry for providing personalized services to users. Previous efforts in user modeling mainly focus on learning a task-specific user representation that is designed for a single task. However, since learning task-specific user representations for every task is infeasible, recent studies introduce the concept of universal user representation, which is a more generalized representation of a user that is relevant to a variety of tasks. Despite their effectiveness, existing approaches for learning universal user representations are impractical in real-world applications due to the data requirement, catastrophic forgetting and the limited learning capability for continually added tasks. In this paper, we propose a novel continual user representation learning method, called TERACON, whose learning capability is not limited as the number of learned tasks increases while capturing the relationship between the tasks. The main idea is to introduce an embedding for each task, i.e., task embedding, which is utilized to generate task-specific soft masks that not only allow the entire model parameters to be updated until the end of training sequence, but also facilitate the relationship between the tasks to be captured. Moreover, we introduce a novel knowledge retention module with pseudo-labeling strategy that successfully alleviates the long-standing problem of continual learning, i.e., catastrophic forgetting. Extensive experiments on public and proprietary real-world datasets demonstrate the superiority and practicality of TERACON. Our code is available at https://github.com/Sein-Kim/TERACON.

翻译：用户建模通过学习用户历史行为将其映射到低维表征空间，为提供个性化服务在工业界引起了广泛关注。现有用户建模方法主要聚焦于为单一任务学习任务特定的用户表征。然而，由于为每个任务学习任务特定用户表征并不可行，近期研究引入了通用用户表征的概念——更通用的用户表征可适用于多种任务。尽管现有通用用户表征学习方法具有有效性，但在实际应用中由于数据需求、灾难性遗忘以及对持续新增任务的学习能力有限等问题而难以落地。本文提出一种名为TERACON的新型持续用户表征学习方法，该方法在学习能力随任务数量增加不受限的同时，能够捕捉任务间关系。核心思想是为每个任务引入任务嵌入（task embedding），用于生成任务特定的软掩码，这不仅使整个模型参数在训练序列结束时仍能持续更新，还促进了任务间关系的捕获。此外，我们引入基于伪标签策略的新型知识保留模块，成功缓解了持续学习中长期存在的灾难性遗忘问题。在公开及专有真实世界数据集上的大量实验证明了TERACON的优越性和实用性。我们的代码已开源：https://github.com/Sein-Kim/TERACON。