Recommender systems have historically developed along two largely independent paradigms: feature interaction models for modeling correlations among multi-field categorical features, and sequential models for capturing user behavior dynamics from historical interaction sequences. Although recent trends attempt to bridge these paradigms within shared backbones, we empirically reveal that naive unifying these two branches may lead to a failure mode of Sequential Collapse Propagation (SCP). That is, the interaction with those dimensionally ill non-sequence fields leads to the dimensional collapse of the sequence features. To overcome this challenge, we propose TokenFormer, a unified recommendation architecture with the following innovations. First, we introduce a Bottom-Full-Top-Sliding (BFTS) attention scheme, which applies full self-attention in the lower layers and shrinking-window sliding attention in the upper layers. Second, we introduce a Non-Linear Interaction Representation (NLIR) that applies one-sided non-linear multiplicative transformations to the hidden states. Extensive experiments on public benchmarks and Tencent's advertising platform demonstrate state-of-the-art performance, while detailed analysis confirm that TokenFormer significantly improves dimensional robustness and representation discriminability under unified modeling.

翻译：推荐系统历史上沿着两条相对独立的范式发展：一是用于建模多场域类别特征间相关性的特征交互模型，二是用于从历史交互序列中捕捉用户行为动态的序贯模型。尽管近期趋势试图在共享骨干网络内桥接这两种范式，但我们通过实验揭示，天真地统一这两个分支可能导致一种称为“序贯坍缩传播”（SCP）的失效模式。即，与那些维度不佳的非序列场域的交互会导致序列特征的维度坍缩。为克服这一挑战，我们提出TokenFormer，一种统一的推荐架构，其创新如下：首先，我们引入一种“底部-全-顶部-滑动”（BFTS）注意力机制，在较低层应用全自注意力，在较高层应用收缩窗口滑动注意力。其次，我们引入一种“非线性交互表示”（NLIR），对隐藏状态施加单侧非线性乘法变换。在公开基准和腾讯广告平台上的大量实验展示了最先进的性能，而详细分析则证实，在统一建模下，TokenFormer显著提升了维度鲁棒性和表示判别性。