End-to-end learning has become a popular method to optimize a constellation shape of a communication system. When the channel model is differentiable, end-to-end learning can be applied with conventional backpropagation algorithm for optimization of the shape. A variety of optimization algorithms have also been developed for end-to-end learning over a non-differentiable channel model. In this paper, we compare gradient-free optimization method based on the cubature Kalman filter, model-free optimization and backpropagation for end-to-end learning on a fiber-optic channel modeled by the split-step Fourier method. The results indicate that the gradient-free optimization algorithms provide a decent replacement to backpropagation in terms of performance at the expense of computational complexity. Furthermore, the quantization problem of finite bit resolution of the digital-to-analog and analog-to-digital converters is addressed and its impact on geometrically shaped constellations is analysed. Here, the results show that when optimizing a constellation with respect to mutual information, a minimum number of quantization levels is required to achieve shaping gain. For generalized mutual information, the gain is maintained throughout all of the considered quantization levels. Also, the results implied that the autoencoder can adapt the constellation size to the given channel conditions.

翻译：端到端学习已成为优化通信系统星座形状的流行方法。当信道模型可微时，可结合传统反向传播算法对星座形状进行端到端学习优化。针对不可微信道模型的端到端学习，也已发展出多种优化算法。本文比较了基于立方体卡尔曼滤波的无梯度优化方法、无模型优化及反向传播算法在采用分步傅里叶方法建模的光纤信道端到端学习中的性能。结果表明，无梯度优化算法在性能上能以计算复杂度为代价有效替代反向传播。进一步地，本文研究了数模转换器与模数转换器有限比特分辨率的量化问题，并分析了其对几何整形星座的影响。结果表明，当以互信息为目标优化星座时，需满足最小量化级数要求以实现整形增益；而对于广义互信息，增益在所有考虑量化级数下均得以保持。此外，结果暗示自编码器能根据给定信道条件自适应调整星座尺寸。