Transformer-based CTR models face a growing bottleneck at the residual connection: under Pre-Norm, early user-interest signals are diluted layer by layer; the identity skip cannot forget stale interests; and each layer sees only its immediate predecessor, losing long-range cross-layer dependencies. Recent attention-based residual variants (AttnRes) address parts of this in language models, but drop the protective identity skip and have not been tried in recommendation. Drawing on Dual Path Networks (DPN) and the HORNN view of residuals, we present DeRes, which routes each layer through two parallel paths -- an Identity residual path that preserves first-order feature reuse and gradient flow, and a Block Attention Residual path that attends over compressed outputs of all earlier blocks for high-order recall. A vector-wise gate decides, per hidden dimension, the weight given to each path. We further propose Pointwise AttnRes, replacing the Softmax in the cross-layer attention with SiLU so that multiple past blocks can be activated simultaneously and irrelevant ones receive negative (forgetting) weights -- better aligned with CTR's parallel multi-interest patterns. On a large-scale industrial dataset (331M interactions from a major social-media platform), Criteo (45M), and Avazu (40M), DeRes outperforms twelve baselines including OneTrans, TokenMixer-Large, UniMixer, mHC, and AttnRes, achieving up to +0.32% AUC at under 5% extra FLOPs. Beyond a single operating point, DeRes fits a markedly steeper compute-AUC scaling law (gamma=0.118 vs. 0.071 for OneTrans, a 1.66x gap), so an 8-layer DeRes matches a 16-layer OneTrans -- about 2x compute saving at equivalent AUC. Ablations confirm that the dual-path design outperforms either single path, Identity beats learnable residuals, and SiLU beats Softmax.

翻译：基于Transformer的点击率预测模型在残差连接处面临日益严峻的瓶颈：在前置归一化下，早期用户兴趣信号逐层衰减；恒等跳跃连接无法遗忘陈旧兴趣；每层仅感知其直接前驱层，丢失了远程跨层依赖关系。近期基于注意力机制的残差变体（AttnRes）虽在语言模型中部分解决了上述问题，但舍弃了保护性恒等跳跃连接，且尚未在推荐系统中得到验证。借鉴双路径网络（DPN）和HORNN对残差的视角，我们提出DeRes，该模型通过两条并行路径路由每一层——保留一阶特征复用和梯度流的恒等残差路径，以及通过跨层注意力机制关注所有先前模块压缩输出以实现高阶回忆的模块注意力残差路径。一个向量级门控机制按隐藏维度决定每条路径的权重。我们进一步提出逐点注意力残差（Pointwise AttnRes），将跨层注意力中的Softmax替换为SiLU，使得多个历史模块可被同时激活，不相关模块获得负（遗忘）权重——更契合CTR任务中并行多兴趣模式。在来自某主流社交媒体平台的大规模工业数据集（3.31亿交互）、Criteo（4500万）和Avazu（4000万）上，DeRes超越包括OneTrans、TokenMixer-Large、UniMixer、mHC和AttnRes在内的十二个基线模型，在额外计算量低于5%的条件下AUC最高提升0.32%。超越单一操作点层面，DeRes拟合出显著更陡峭的计算量-AUC缩放定律（伽马=0.118 vs. OneTrans的0.071，1.66倍差距），因此8层DeRes可与16层OneTrans性能匹敌——在同等AUC下节省约2倍计算量。消融实验证实，双路径设计优于任一单一路径，恒等跳跃连接优于可学习残差，SiLU优于Softmax。