Machine learning based approaches are being increasingly used for designing decoders for next generation communication systems. One widely used framework is neural belief propagation (NBP), which unfolds the belief propagation (BP) iterations into a deep neural network and the parameters are trained in a data-driven manner. NBP decoders have been shown to improve upon classical decoding algorithms. In this paper, we investigate the generalization capabilities of NBP decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s). We present new theoretical results which bound this gap and show the dependence on the decoder complexity, in terms of code parameters (blocklength, message length, variable/check node degrees), decoding iterations, and the training dataset size. Results are presented for both regular and irregular parity-check matrices. To the best of our knowledge, this is the first set of theoretical results on generalization performance of neural network based decoders. We present experimental results to show the dependence of generalization gap on the training dataset size, and decoding iterations for different codes.

翻译：基于机器学习的方法正越来越多地用于设计下一代通信系统的解码器。其中一种广泛使用的框架是神经信念传播（NBP），它将信念传播（BP）迭代展开为深度神经网络，并以数据驱动的方式训练参数。研究表明，NBP解码器能够改进经典解码算法。本文研究了NBP解码器的泛化能力。具体而言，解码器的泛化差距是经验误码率与期望误码率之间的差异。我们提出了新的理论结果，界定了这一差距，并展示了其对解码器复杂性的依赖关系，该复杂性通过码参数（码长、信息长度、变量/校验节点度数）、解码迭代次数和训练数据集大小来表征。结果同时针对规则和非规则校验矩阵给出。据我们所知，这是关于基于神经网络的解码器泛化性能的首组理论结果。我们通过实验展示了不同码字下泛化差距对训练数据集大小和解码迭代次数的依赖性。