Selective State Space Models (SSMs), notably Mamba, employ diagonal state transitions that limit both memory retention and bilinear computational capacity. We propose a factorized bilinear input modulation that augments the SSM with a state-input product, interpretable as a finite-dimensional Koopman bilinear form. After introducing a shared state across channels (Coupled SSM), the modulation admits two implementations. Coupled Bilinear Input Modulation (Coupled-BIM) retains the full bilinear product at the cost of sequential computation, while Coupled Gated Modulation (Coupled-GM) linearizes it into a gate modulation that is compatible with the parallel scan. Experiments on a multiple input-delay pendulum (memory retention) and NARMA-10 (bilinear computation) reveal a clear dissociation. Coupled-GM substantially improves memory retention but not bilinear computation, while Coupled-BIM improves both. A pathway ablation confirms that the two downstream routes of the bilinear signal serve complementary roles. The improvement is statistically robust, with Coupled-BIM consistently outperforming all other variants on bilinear computation. Furthermore, only Coupled-BIM benefits from increasing the SSM state dimension, while coupling or gate modulation alone show no improvement, establishing the bilin-ear mechanism as uniquely capable of exploiting larger state spaces.
翻译:选择性状态空间模型(SSMs),特别是Mamba,采用对角状态转移限制了记忆保持和双线性计算能力。本文提出一种分解式双线性输入调制方法,通过引入状态-输入乘积来增强SSM,该乘积可解释为有限维库普曼双线性形式。在跨通道引入共享状态(耦合SSM)后,该调制方法有两种实现方式:耦合双线性输入调制(Coupled-BIM)保留完整双线性乘积但需顺序计算;而耦合门控调制(Coupled-GM)将其线性化为与并行扫描兼容的门控调制。在多输入延迟摆(记忆保持)和NARMA-10(双线性计算)上的实验揭示了明确的功能分离:Coupled-GM显著提升记忆保持但对双线性计算无改善,而Coupled-BIM对两者均有提升。路径消融实验证实双线性信号的两条下游通路具有互补作用。统计结果表明改进具有稳健性,Coupled-BIM在双线性计算上持续优于所有其他变体。此外,仅Coupled-BIM能从增大SSM状态维度中获益,而单独采用耦合或门控调制均无改善,这确立了双线性机制是唯一能有效利用更大状态空间的机制。