Early neural channel coding approaches leveraged dense neural networks with one-hot encodings to design adaptive encoder-decoder pairs, improving block error rate (BLER) and automating the design process. However, these methods struggled with scalability as the size of message sets and block lengths increased. TurboAE addressed this challenge by focusing on bit-sequence inputs rather than symbol-level representations, transforming the scalability issue associated with large message sets into a sequence modeling problem. While recurrent neural networks (RNNs) were a natural fit for sequence processing, their reliance on sequential computations made them computationally expensive and inefficient for long sequences. As a result, TurboAE adopted convolutional network blocks, which were faster to train and more scalable, but lacked the sequential modeling advantages of RNNs. Recent advances in efficient RNN architectures, such as minGRU and minLSTM, and structured state space models (SSMs) like S4 and S6, overcome these limitations by significantly reducing memory and computational overhead. These models enable scalable sequence processing, making RNNs competitive for long-sequence tasks. In this work, we revisit RNNs for Turbo autoencoders by integrating the lightweight minGRU model with a Mamba block from SSMs into a parallel Turbo autoencoder framework. Our results demonstrate that this hybrid design matches the performance of convolutional network-based Turbo autoencoder approaches for short sequences while significantly improving scalability and training efficiency for long block lengths. This highlights the potential of efficient RNNs in advancing neural channel coding for long-sequence scenarios.

翻译：早期的神经信道编码方法利用具有独热编码的密集神经网络来设计自适应编码器-解码器对，从而改善误块率（BLER）并自动化设计过程。然而，随着消息集大小和块长度的增加，这些方法在可扩展性方面面临挑战。TurboAE通过关注比特序列输入而非符号级表示，将大规模消息集相关的可扩展性问题转化为序列建模问题，从而应对了这一挑战。虽然循环神经网络（RNNs）天然适合序列处理，但其对顺序计算的依赖使得它们在计算上成本高昂，对于长序列效率低下。因此，TurboAE采用了卷积网络块，这些块训练速度更快、可扩展性更强，但缺乏RNNs的序列建模优势。近期在高效RNN架构（如minGRU和minLSTM）以及结构化状态空间模型（SSMs，如S4和S6）方面的进展，通过显著降低内存和计算开销克服了这些限制。这些模型实现了可扩展的序列处理，使得RNNs在长序列任务中具有竞争力。在本工作中，我们通过将轻量级minGRU模型与SSMs中的Mamba块集成到并行Turbo自编码器框架中，重新审视了RNNs在Turbo自编码器中的应用。我们的结果表明，这种混合设计在短序列上匹配了基于卷积网络的Turbo自编码器方法的性能，同时在长块长度下显著提高了可扩展性和训练效率。这凸显了高效RNNs在推进长序列场景下神经信道编码方面的潜力。