Modern large-scale recommendation systems rely heavily on user interaction history sequences to enhance the model performance. The advent of large language models and sequential modeling techniques, particularly transformer-like architectures, has led to significant advancements recently (e.g., HSTU, SIM, and TWIN models). While scaling to ultra-long user histories (10k to 100k items) generally improves model performance, it also creates significant challenges on latency, queries per second (QPS) and GPU cost in industry-scale recommendation systems. Existing models do not adequately address these industrial scalability issues. In this paper, we propose a novel two-stage modeling framework, namely VIrtual Sequential Target Attention (VISTA), which decomposes traditional target attention from a candidate item to user history items into two distinct stages: (1) user history summarization into a few hundred tokens; followed by (2) candidate item attention to those tokens. These summarization token embeddings are then cached in storage system and then utilized as sequence features for downstream model training and inference. This novel design for scalability enables VISTA to scale to lifelong user histories (up to one million items) while keeping downstream training and inference costs fixed, which is essential in industry. Our approach achieves significant improvements in offline and online metrics and has been successfully deployed on an industry leading recommendation platform serving billions of users.

翻译：现代大规模推荐系统高度依赖用户交互历史序列来提升模型性能。大型语言模型与序列建模技术（特别是类Transformer架构）的出现，近期已推动该领域取得显著进展（例如HSTU、SIM和TWIN模型）。尽管扩展至超长用户历史（1万至10万条项目）通常能提升模型性能，但这在工业级推荐系统中也带来了延迟、每秒查询量（QPS）和GPU成本的重大挑战。现有模型未能充分解决这些工业可扩展性问题。本文提出一种新颖的两阶段建模框架——虚拟序列目标注意力（VISTA），该框架将传统从候选项目到用户历史项目的目标注意力分解为两个独立阶段：（1）将用户历史摘要为数百个标记；（2）候选项目对这些标记的注意力计算。这些摘要标记嵌入随后被缓存在存储系统中，并作为序列特征用于下游模型的训练与推理。这种创新的可扩展性设计使VISTA能够扩展至终身用户历史（高达一百万条项目），同时保持下游训练与推理成本固定，这在工业场景中至关重要。我们的方法在离线与在线指标上均取得显著提升，并已成功部署于服务数十亿用户的行业领先推荐平台。