Distributed nonconvex optimization underpins key functionalities of numerous distributed systems, ranging from power systems, smart buildings, cooperative robots, vehicle networks to sensor networks. Recently, it has also merged as a promising solution to handle the enormous growth in data and model sizes in deep learning. A fundamental problem in distributed nonconvex optimization is avoiding convergence to saddle points, which significantly degrade optimization accuracy. We discover that the process of quantization, which is necessary for all digital communications, can be exploited to enable saddle-point avoidance. More specifically, we propose a stochastic quantization scheme and prove that it can effectively escape saddle points and ensure convergence to a second-order stationary point in distributed nonconvex optimization. With an easily adjustable quantization granularity, the approach allows a user to control the number of bits sent per iteration and, hence, to aggressively reduce the communication overhead. Numerical experimental results using distributed optimization and learning problems on benchmark datasets confirm the effectiveness of the approach.

翻译：分布式非凸优化支撑着众多分布式系统的关键功能，涵盖电力系统、智能建筑、协作机器人、车辆网络及传感器网络等。最近，它也成为处理深度学习中海量数据与模型规模增长的有前景的解决方案。分布式非凸优化中的一个根本性问题在于避免收敛至鞍点，这会显著降低优化精度。我们发现，所有数字通信所必需的量化过程，可被利用来实现鞍点规避。具体而言，我们提出了一种随机量化方案，并证明该方案能有效逃离鞍点，确保在分布式非凸优化中收敛至二阶稳定点。该方法通过易于调节的量化粒度，使用户能够控制每次迭代传输的比特数，从而大幅降低通信开销。基于基准数据集上的分布式优化与学习问题的数值实验结果，验证了该方法的有效性。