End-to-end learning has become a popular method to optimize a constellation shape of a communication system. When the channel model is differentiable, end-to-end learning can be applied with conventional backpropagation algorithm for optimization of the shape. A variety of optimization algorithms have also been developed for end-to-end learning over a non-differentiable channel model. In this paper, we compare gradient-free optimization method based on the cubature Kalman filter, model-free optimization and backpropagation for end-to-end learning on a fiber-optic channel modeled by the split-step Fourier method. The results indicate that the gradient-free optimization algorithms provide a decent replacement to backpropagation in terms of performance at the expense of computational complexity. Furthermore, the quantization problem of finite bit resolution of the digital-to-analog and analog-to-digital converters is addressed and its impact on geometrically shaped constellations is analysed. Here, the results show that when optimizing a constellation with respect to mutual information, a minimum number of quantization levels is required to achieve shaping gain. For generalized mutual information, the gain is maintained throughout all of the considered quantization levels. Also, the results implied that the autoencoder can adapt the constellation size to the given channel conditions.

翻译：端到端学习已成为优化通信系统星座形状的常用方法。当信道模型可微时，可采用传统反向传播算法进行端到端学习以优化星座形状。针对不可微信道模型，研究人员也开发了多种端到端学习优化算法。本文在基于分步傅里叶方法建模的光纤信道上，比较了基于容积卡尔曼滤波的无梯度优化方法、无模型优化及反向传播三种端到端学习方案。结果表明，无梯度优化算法虽以计算复杂度为代价，但其性能可有效替代反向传播方法。此外，本文还研究了数模/模数转换器有限比特分辨率的量化问题，并分析了其对几何星座的影响。研究发现：当以互信息为优化目标时，需满足最小量化级数才能实现成形增益；而以广义互信息为优化目标时，所有量化级数均可保持成形增益。同时，仿真结果表明自编码器能根据信道条件自适应调整星座尺寸。