We study the sorting-based embedding $\beta_{\mathbf A} : \mathbb R^{n \times d} \to \mathbb R^{n \times D}$, $\mathbf X \mapsto {\downarrow}(\mathbf X \mathbf A)$, where $\downarrow$ denotes column wise sorting of matrices. Such embeddings arise in graph deep learning where outputs should be invariant to permutations of graph nodes. Previous work showed that for large enough $D$ and appropriate $\mathbf A$, the mapping $\beta_{\mathbf A}$ is injective, and moreover satisfies a bi-Lipschitz condition. However, two gaps remain: firstly, the optimal size $D$ required for injectivity is not yet known, and secondly, no estimates of the bi-Lipschitz constants of the mapping are known. In this paper, we make substantial progress in addressing both of these gaps. Regarding the first gap, we improve upon the best known upper bounds for the embedding dimension $D$ necessary for injectivity, and also provide a lower bound on the minimal injectivity dimension. Regarding the second gap, we construct matrices $\mathbf A$, so that the bi-Lipschitz distortion of $\beta_{\mathbf A} $ depends quadratically on $n$, and is completely independent of $d$. We also show that the distortion of $\beta_{\mathbf A}$ is necessarily at least in $\Omega(\sqrt{n})$. Finally, we provide similar results for variants of $\beta_{\mathbf A}$ obtained by applying linear projections to reduce the output dimension of $\beta_{\mathbf A}$.
翻译:我们研究基于排序的嵌入映射 $\beta_{\mathbf A} : \mathbb R^{n \times d} \to \mathbb R^{n \times D}$, $\mathbf X \mapsto {\downarrow}(\mathbf X \mathbf A)$,其中 $\downarrow$ 表示对矩阵进行逐列排序。此类嵌入出现在图深度学习中,其中输出应对图节点的置换保持不变。先前的研究表明,对于足够大的 $D$ 和适当的 $\mathbf A$,映射 $\beta_{\mathbf A}$ 是单射的,并且满足双 Lipschitz 条件。然而,仍存在两个未解决的问题:首先,实现单射性所需的最优维度 $D$ 尚不清楚;其次,该映射的双 Lipschitz 常数尚未有估计。本文在解决这两个问题上取得了实质性进展。针对第一个问题,我们改进了实现单射性所需嵌入维度 $D$ 的最佳已知上界,并给出了最小单射维度的下界。针对第二个问题,我们构造了矩阵 $\mathbf A$,使得 $\beta_{\mathbf A}$ 的双 Lipschitz 失真与 $n$ 成二次方依赖关系,且完全独立于 $d$。我们还证明了 $\beta_{\mathbf A}$ 的失真至少为 $\Omega(\sqrt{n})$。最后,我们对 $\beta_{\mathbf A}$ 的变体给出了类似的结果,这些变体通过应用线性投影来降低 $\beta_{\mathbf A}$ 的输出维度。