Machine learning based approaches are being increasingly used for designing decoders for next generation communication systems. One widely used framework is neural belief propagation (NBP), which unfolds the belief propagation (BP) iterations into a deep neural network and the parameters are trained in a data-driven manner. NBP decoders have been shown to improve upon classical decoding algorithms. In this paper, we investigate the generalization capabilities of NBP decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s). We present new theoretical results which bound this gap and show the dependence on the decoder complexity, in terms of code parameters (blocklength, message length, variable/check node degrees), decoding iterations, and the training dataset size. Results are presented for both regular and irregular parity-check matrices. To the best of our knowledge, this is the first set of theoretical results on generalization performance of neural network based decoders. We present experimental results to show the dependence of generalization gap on the training dataset size, and decoding iterations for different codes.

翻译：基于机器学习的方法正越来越多地被用于设计下一代通信系统的解码器。一种广泛使用的框架是神经置信传播（NBP），它将置信传播（BP）迭代展开为深度神经网络，并以数据驱动的方式训练参数。研究表明，NBP解码器优于经典解码算法。本文研究了NBP解码器的泛化能力。具体而言，解码器的泛化差距是经验误码率与期望误码率之间的差异。我们提出了新的理论结果，对该差距进行了界定，并展示了其与解码器复杂度的依赖关系，涉及码参数（码长、信息长度、变量/校验节点度数）、解码迭代次数以及训练数据集大小。结果针对规则和非规则校验矩阵均给出。据我们所知，这是关于基于神经网络解码器泛化性能的首套理论结果。我们通过实验展示了不同码型下泛化差距对训练数据集大小和解码迭代次数的依赖性。