We present LLaTTE (LLM-Style Latent Transformers for Temporal Events), a scalable transformer architecture for production ads recommendation. Through systematic experiments, we demonstrate that sequence modeling in recommendation systems follows predictable power-law scaling similar to LLMs. Crucially, we find that semantic features bend the scaling curve: they are a prerequisite for scaling, enabling the model to effectively utilize the capacity of deeper and longer architectures. To realize the benefits of continued scaling under strict latency constraints, we introduce a two-stage architecture that offloads the heavy computation of large, long-context models to an asynchronous upstream user model. We demonstrate that upstream improvements transfer predictably to downstream ranking tasks. Deployed as the largest user model at Meta, this multi-stage framework drives a 4.3\% conversion uplift on Facebook Feed and Reels with minimal serving overhead, establishing a practical blueprint for harnessing scaling laws in industrial recommender systems.

翻译：我们提出LLaTTE（面向时序事件的LLM风格潜在Transformer），一种用于生产环境广告推荐的可扩展Transformer架构。通过系统实验，我们证明了推荐系统中的序列建模遵循与LLM类似的、可预测的幂律缩放规律。关键的是，我们发现语义特征会改变缩放曲线的形态：它们是实现缩放的前提条件，使模型能够有效利用更深、更长架构的容量。为了在严格延迟约束下实现持续缩放的优势，我们引入了一种两阶段架构，将大型长上下文模型的重计算任务卸载到异步上游用户模型中。我们证明了上游模型的改进可以按可预测的方式迁移到下游排序任务中。作为Meta部署的最大规模用户模型，该多阶段框架在Facebook Feed和Reels上实现了4.3%的转化率提升，且服务开销极小，为在工业推荐系统中利用缩放定律建立了一个实用的蓝图。