This paper presents a novel unified theoretical framework for understanding Transformer architectures by integrating Partial Differential Equations (PDEs), Neural Information Flow Theory, and Information Bottleneck Theory. We model Transformer information dynamics as a continuous PDE process, encompassing diffusion, self-attention, and nonlinear residual components. Our comprehensive experiments across image and text modalities demonstrate that the PDE model effectively captures key aspects of Transformer behavior, achieving high similarity (cosine similarity > 0.98) with Transformer attention distributions across all layers. While the model excels in replicating general information flow patterns, it shows limitations in fully capturing complex, non-linear transformations. This work provides crucial theoretical insights into Transformer mechanisms, offering a foundation for future optimizations in deep learning architectural design. We discuss the implications of our findings, potential applications in model interpretability and efficiency, and outline directions for enhancing PDE models to better mimic the intricate behaviors observed in Transformers, paving the way for more transparent and optimized AI systems.

翻译：本文提出了一种新颖的统一理论框架，通过整合偏微分方程（PDEs）、神经信息流理论与信息瓶颈理论来理解Transformer架构。我们将Transformer的信息动力学建模为一个连续的PDE过程，涵盖扩散、自注意力与非线性残差分量。我们在图像与文本模态上的综合实验表明，该PDE模型能有效捕捉Transformer行为的关键方面，在所有层中与Transformer注意力分布实现了高度相似性（余弦相似度 > 0.98）。尽管该模型在复现一般信息流模式方面表现优异，但在完全捕捉复杂的非线性变换方面仍存在局限。本工作为Transformer机制提供了重要的理论洞见，为未来深度学习架构设计的优化奠定了基础。我们讨论了研究结果的启示、在模型可解释性与效率方面的潜在应用，并概述了增强PDE模型以更好地模拟Transformer中观察到的复杂行为的方向，从而为构建更透明、更优化的AI系统铺平道路。