In this paper, we analyze the computational limitations of Mamba and State-space Models (SSMs) by using the circuit complexity framework. Despite Mamba's stateful design and recent attention as a strong candidate to outperform Transformers, we have demonstrated that both Mamba and SSMs with $\mathrm{poly}(n)$-precision and constant-depth layers reside within the $\mathsf{DLOGTIME}$-uniform $\mathsf{TC}^0$ complexity class. This result indicates Mamba has the same computational capabilities as Transformer theoretically, and it cannot solve problems like arithmetic formula problems, boolean formula value problems, and permutation composition problems if $\mathsf{TC}^0 \neq \mathsf{NC}^1$. Therefore, it challenges the assumption Mamba is more computationally expressive than Transformers. Our contributions include rigorous proofs showing that Selective SSM and Mamba architectures can be simulated by $\mathsf{DLOGTIME}$-uniform $\mathsf{TC}^0$ circuits, and they cannot solve problems outside $\mathsf{TC}^0$.

翻译：本文通过电路复杂度框架分析了Mamba与状态空间模型（SSMs）的计算局限性。尽管Mamba采用状态化设计且近期被视为超越Transformer的有力候选模型，我们证明了具有$\mathrm{poly}(n)$精度和恒定深度层的Mamba与SSMs均属于$\mathsf{DLOGTIME}$-均匀$\mathsf{TC}^0$复杂度类。该结果表明Mamba在理论上具有与Transformer相同的计算能力，且若$\mathsf{TC}^0 \neq \mathsf{NC}^1$成立，则其无法解决算术公式问题、布尔公式求值问题及置换组合问题。这挑战了“Mamba比Transformer具有更强计算表达能力”的假设。我们的贡献包括严格证明选择性SSM与Mamba架构可由$\mathsf{DLOGTIME}$-均匀$\mathsf{TC}^0$电路模拟，且二者均无法求解$\mathsf{TC}^0$类之外的问题。