Despite the recent popularity of neural network-based solvers for optimal transport (OT), there is no standard quantitative way to evaluate their performance. In this paper, we address this issue for quadratic-cost transport -- specifically, computation of the Wasserstein-2 distance, a commonly-used formulation of optimal transport in machine learning. To overcome the challenge of computing ground truth transport maps between continuous measures needed to assess these solvers, we use input-convex neural networks (ICNN) to construct pairs of measures whose ground truth OT maps can be obtained analytically. This strategy yields pairs of continuous benchmark measures in high-dimensional spaces such as spaces of images. We thoroughly evaluate existing optimal transport solvers using these benchmark measures. Even though these solvers perform well in downstream tasks, many do not faithfully recover optimal transport maps. To investigate the cause of this discrepancy, we further test the solvers in a setting of image generation. Our study reveals crucial limitations of existing solvers and shows that increased OT accuracy does not necessarily correlate to better results downstream.

翻译：尽管最近以神经网络为基础的求解器对最佳运输(OT)很受欢迎,但目前没有标准的数量方法来评价它们的业绩。在本文中,我们讨论四维成本运输的问题,具体地说,计算瓦塞斯坦-2号距离,这是在机器学习中常用的最佳运输方法。为了克服计算地面真象运输图的挑战,在评估这些求解器所需的连续措施之间,我们使用输入-convex神经网络(ICNNN)来构建一对措施,其地面真象OT地图可以分析获得。这个战略在高维空间(如图像空间)产生连续的基准措施。我们用这些基准措施彻底评估现有的最佳运输求解器。尽管这些解答器在下游任务中表现良好,但许多人并不忠实地恢复最佳运输图。为了调查这一差异的原因,我们进一步测试在图像生成过程中的解答器。我们的研究揭示了现有解算器的关键局限性,并表明提高OT精确度并不一定与下游更好的结果相关。