Despite being one of the earliest neural network layers, the Multilayer Perceptron (MLP) is arguably one of the least understood parts of the transformer architecture due to its dense computation and lack of easy visualization. This paper seeks to understand the MLP layers in dense LLM models by hypothesizing that these layers secretly approximately perform a sparse computation -- namely, that they can be well approximated by sparsely-activating Mixture of Experts (MoE) layers. Our hypothesis is based on a novel theoretical connection between MoE models and Sparse Autoencoder (SAE) structure in activation space. We empirically validate the hypothesis on pretrained LLMs, and demonstrate that the activation distribution matters -- these results do not hold for Gaussian data, but rather rely crucially on structure in the distribution of neural network activations. Our results shine light on a general principle at play in MLP layers inside LLMs, and give an explanation for the effectiveness of modern MoE-based transformers. Additionally, our experimental explorations suggest new directions for more efficient MoE architecture design based on low-rank routers.

翻译：尽管多层感知机（MLP）是最早的神经网络层之一，但由于其密集的计算特性且难以可视化，它可以说是Transformer架构中最不为人理解的组成部分之一。本文旨在通过提出一个假设来理解稠密大型语言模型中的MLP层：这些层实际上近似执行着稀疏计算——即它们可以被稀疏激活的专家混合（MoE）层很好地近似。我们的假设基于MoE模型与激活空间中稀疏自编码器（SAE）结构之间新颖的理论联系。我们在预训练的大型语言模型上实证验证了这一假设，并证明激活分布至关重要——这些结果在高斯数据上并不成立，而是严重依赖于神经网络激活分布的结构特性。我们的研究结果揭示了大型语言模型中MLP层运作的普遍原理，并为基于MoE的现代Transformer架构的有效性提供了解释。此外，我们的实验探索为基于低秩路由器的更高效MoE架构设计提出了新的研究方向。