DNNs have gained widespread adoption in feature interaction recommendation models. However, there has been a longstanding debate on their roles. On one hand, some works claim that DNNs possess the ability to implicitly capture high-order feature interactions. Conversely, recent studies have highlighted the limitations of DNNs in effectively learning dot products, specifically second-order interactions, let alone higher-order interactions. In this paper, we present a novel perspective to understand the effectiveness of DNNs: their impact on the dimensional robustness of the representations. In particular, we conduct extensive experiments involving both parallel DNNs and stacked DNNs. Our evaluation encompasses an overall study of complete DNN on two feature interaction models, alongside a fine-grained ablation analysis of components within DNNs. Experimental results demonstrate that both parallel and stacked DNNs can effectively mitigate the dimensional collapse of embeddings. Furthermore, a gradient-based theoretical analysis, supported by empirical evidence, uncovers the underlying mechanisms of dimensional collapse.

翻译：深度神经网络（DNN）已在特征交互推荐模型中广泛应用。然而，学界对其作用长期存在争论：一方面，部分研究声称DNN具备隐式捕获高阶特征交互的能力；另一方面，近期研究指出DNN在有效学习点积（即二阶交互）方面存在局限，更遑论高阶交互。本文提出理解DNN有效性的新视角：其对表征维度鲁棒性的影响。具体而言，我们针对并行DNN与堆叠DNN开展了系统性实验。评估工作涵盖两种特征交互模型上完整DNN的总体研究，以及DNN内部组件的细粒度消融分析。实验结果表明，并行DNN与堆叠DNN均能有效缓解嵌入向量的维度塌缩。此外，基于梯度的理论分析（辅以实证证据）揭示了维度塌缩的潜在机制。