We propose a method to compare and visualise sentence encoders at scale by creating a map of encoders where each sentence encoder is represented in relation to the other sentence encoders. Specifically, we first represent a sentence encoder using an embedding matrix of a sentence set, where each row corresponds to the embedding of a sentence. Next, we compute the Pairwise Inner Product (PIP) matrix for a sentence encoder using its embedding matrix. Finally, we create a feature vector for each sentence encoder reflecting its Quantum Relative Entropy (QRE) with respect to a unit base encoder. We construct a map of encoders covering 1101 publicly available sentence encoders, providing a new perspective of the landscape of the pre-trained sentence encoders. Our map accurately reflects various relationships between encoders, where encoders with similar attributes are proximally located on the map. Moreover, our encoder feature vectors can be used to accurately infer downstream task performance of the encoders, such as in retrieval and clustering tasks, demonstrating the faithfulness of our map.

翻译：我们提出了一种大规模比较和可视化句子编码器的方法，通过构建编码器地图，使每个句子编码器相对于其他编码器得以呈现。具体而言，我们首先使用句子集的嵌入矩阵表示句子编码器，其中每一行对应一个句子的嵌入向量。接着，我们利用该嵌入矩阵计算每个句子编码器的成对内积矩阵。最后，我们为每个句子编码器构建一个特征向量，该向量反映其相对于单位基编码器的量子相对熵。我们构建的编码器地图涵盖了1101个公开可用的句子编码器，为预训练句子编码器的生态格局提供了新的观察视角。该地图精确反映了编码器间的多种关联关系，具有相似属性的编码器在地图上位置相邻。此外，我们的编码器特征向量可用于准确推断编码器在下游任务（如检索和聚类任务）中的性能表现，从而验证了地图的可靠性。