Recurrent neural networks (RNNs) and transformers have been shown to be Turing-complete, but this result assumes infinite precision in their hidden representations, positional encodings for transformers, and unbounded computation time in general. In practical applications, however, it is crucial to have real-time models that can recognize Turing complete grammars in a single pass. To address this issue and to better understand the true computational power of artificial neural networks (ANNs), we introduce a new class of recurrent models called the neural state Turing machine (NSTM). The NSTM has bounded weights and finite-precision connections and can simulate any Turing Machine in real-time. In contrast to prior work that assumes unbounded time and precision in weights, to demonstrate equivalence with TMs, we prove that a $13$-neuron bounded tensor RNN, coupled with third-order synapses, can model any TM class in real-time. Furthermore, under the Markov assumption, we provide a new theoretical bound for a non-recurrent network augmented with memory, showing that a tensor feedforward network with $25$th-order finite precision weights is equivalent to a universal TM.

翻译：递归神经网络（RNN）与Transformer已被证明具有图灵完备性，但这一结论依赖于其隐层表示的无限精度、Transformer的位置编码以及无限计算时间。然而在实际应用中，能够单次通过实时识别图灵完备文法的模型至关重要。为解决该问题并深入理解人工神经网络（ANNs）的真实计算能力，我们提出一类新型递归模型——神经状态图灵机（NSTM）。NSTM具有有界权重与有限精度连接，可实时模拟任意图灵机。与先前假设权重具有无限精度与时间的图灵等价性研究不同，我们证明一个配备三阶突触的13神经元有界张量循环神经网络即可实时建模任何类别的图灵机。此外，在马尔可夫假设下，我们为带记忆的非递归网络提供了新的理论界限，表明具有25阶有限精度权重的张量前馈网络等价于通用图灵机。