Sorting is a fundamental operation of all computer systems, having been a long-standing significant research topic. Beyond the problem formulation of traditional sorting algorithms, we consider sorting problems for more abstract yet expressive inputs, e.g., multi-digit images and image fragments, through a neural sorting network. To learn a mapping from a high-dimensional input to an ordinal variable, the differentiability of sorting networks needs to be guaranteed. In this paper we define a softening error by a differentiable swap function, and develop an error-free swap function that holds a non-decreasing condition and differentiability. Furthermore, a permutation-equivariant Transformer network with multi-head attention is adopted to capture dependency between given inputs and also leverage its model capacity with self-attention. Experiments on diverse sorting benchmarks show that our methods perform better than or comparable to baseline methods.

翻译：排序是所有计算机系统的核心操作，长期以来一直是重要的研究课题。除传统排序算法的问题定义外，我们通过神经排序网络进一步考虑更抽象且表达性更强的输入（例如多位数图像与图像片段）的排序问题。为学习从高维输入到序数变量的映射，需保证排序网络的可微性。本文通过可微交换函数定义软化误差，并开发了满足非递减条件且保持可微性的无误差交换函数。此外，采用基于多头注意力的置换等变Transformer网络，以捕获给定输入间的依赖关系，并利用其自注意力机制的模型容量。在多种排序基准测试上的实验表明，我们的方法性能优于或可比肩基线方法。