Neural networks embed the geometric structure of a data manifold lying in a high-dimensional space into latent representations. Ideally, the distribution of the data points in the latent space should depend only on the task, the data, the loss, and other architecture-specific constraints. However, factors such as the random weights initialization, training hyperparameters, or other sources of randomness in the training phase may induce incoherent latent spaces that hinder any form of reuse. Nevertheless, we empirically observe that, under the same data and modeling choices, the angles between the encodings within distinct latent spaces do not change. In this work, we propose the latent similarity between each sample and a fixed set of anchors as an alternative data representation, demonstrating that it can enforce the desired invariances without any additional training. We show how neural architectures can leverage these relative representations to guarantee, in practice, invariance to latent isometries and rescalings, effectively enabling latent space communication: from zero-shot model stitching to latent space comparison between diverse settings. We extensively validate the generalization capability of our approach on different datasets, spanning various modalities (images, text, graphs), tasks (e.g., classification, reconstruction) and architectures (e.g., CNNs, GCNs, transformers).

翻译：神经网络将高维空间中数据流形的几何结构嵌入到潜在表示中。理想情况下，潜在空间中数据点的分布应仅取决于任务、数据、损失函数以及其他架构特定约束。然而，随机权重初始化、训练超参数或训练阶段的其他随机性因素可能导致不一致的潜在空间，从而阻碍任何形式的复用。尽管如此，我们通过经验观察发现：在相同数据和建模选择的前提下，不同潜在空间中编码之间的角度不会发生改变。本文提出将每个样本与固定锚点集之间的潜在相似性作为替代数据表示，证明该方法无需额外训练即可实现所需的恒等性。我们展示了神经架构如何利用这些相对表示，在实践中保证对潜在等距变换和尺度缩放的恒等性，从而有效实现潜在空间通信：从零样本模型拼接到不同设置间的潜在空间比较。我们在不同数据集上广泛验证了方法的泛化能力，涵盖多种模态（图像、文本、图）、任务（如分类、重建）和架构（如CNN、GCN、Transformer）。