The use of pretrained embeddings has become widespread in modern e-commerce machine learning (ML) systems. In practice, however, we have encountered several key issues when using pretrained embedding in a real-world production system, many of which cannot be fully explained by current knowledge. Unfortunately, we find that there is a lack of a thorough understanding of how pre-trained embeddings work, especially their intrinsic properties and interactions with downstream tasks. Consequently, it becomes challenging to make interactive and scalable decisions regarding the use of pre-trained embeddings in practice. Our investigation leads to two significant discoveries about using pretrained embeddings in e-commerce applications. Firstly, we find that the design of the pretraining and downstream models, particularly how they encode and decode information via embedding vectors, can have a profound impact. Secondly, we establish a principled perspective of pre-trained embeddings via the lens of kernel analysis, which can be used to evaluate their predictability, interactively and scalably. These findings help to address the practical challenges we faced and offer valuable guidance for successful adoption of pretrained embeddings in real-world production. Our conclusions are backed by solid theoretical reasoning, benchmark experiments, as well as online testings.

翻译：使用预训练嵌入已成为现代电子商务机器学习系统中的普遍做法。然而，在实际生产系统中，我们遇到了多个关键问题，其中许多无法通过现有知识完全解释。遗憾的是，我们发现目前对预训练嵌入的工作原理缺乏深入理解，尤其是其内在属性及与下游任务的交互机制。因此，在实践中有序且可扩展地决策预训练嵌入的使用变得极具挑战性。我们的研究针对电子商务应用中预训练嵌入的使用取得了两项重要发现。首先，预训练模型与下游模型的设计，特别是它们通过嵌入向量编码与解码信息的方式，会产生深远影响。其次，我们通过核分析的视角建立了评估预训练嵌入的原则性框架，该框架可交互且可扩展地评估其可预测性。这些发现有助于解决我们面临的实际挑战，并为预训练嵌入在真实生产系统中的成功应用提供宝贵指导。我们的结论得到了严谨理论推导、基准实验以及在线测试的充分支持。