There is a niche of companies responsible for intermediating the purchase of large batches of varied products for other companies, for which the main challenge is to perform product description standardization, i.e., matching an item described by a client with a product described in a catalog. The problem is complex since the client's product description may be: (1) potentially noisy; (2) short and uninformative (e.g., missing information about model and size); and (3) cross-language. In this paper, we formalize this problem as a ranking task: given an initial client product specification (query), return the most appropriate standardized descriptions (response). In this paper, we propose TPDR, a two-step Transformer-based Product and Class Description Retrieval method that is able to explore the semantic correspondence between IS and SD, by exploiting attention mechanisms and contrastive learning. First, TPDR employs the transformers as two encoders sharing the embedding vector space: one for encoding the IS and another for the SD, in which corresponding pairs (IS, SD) must be close in the vector space. Closeness is further enforced by a contrastive learning mechanism leveraging a specialized loss function. TPDR also exploits a (second) re-ranking step based on syntactic features that are very important for the exact matching (model, dimension) of certain products that may have been neglected by the transformers. To evaluate our proposal, we consider 11 datasets from a real company, covering different application contexts. Our solution was able to retrieve the correct standardized product before the 5th ranking position in 71% of the cases and its correct category in the first position in 80% of the situations. Moreover, the effectiveness gains over purely syntactic or semantic baselines reach up to 3.7 times, solving cases that none of the approaches in isolation can do by themselves.
翻译:存在一类专为其他公司批量采购多样化产品的中间商企业,其核心挑战在于实现产品描述标准化——即匹配客户描述条目与产品目录中的标准化描述。该问题的复杂性源于客户产品描述可能具备以下特征:(1) 潜在噪声干扰;(2) 简短且信息不完整(如缺失型号和尺寸信息);(3) 跨语言表述。本文将此类问题形式化为排序任务:给定初始客户产品规格(查询),返回最匹配的标准化描述(响应)。为此,我们提出TPDR——一种基于Transformer的两步式产品与类别描述检索方法,通过注意力机制与对比学习探索初始规格与标准化描述之间的语义对应关系。首先,TPDR采用两个共享嵌入向量空间的Transformer编码器,分别对初始规格与标准化描述进行编码,使匹配对在向量空间中保持邻近。该邻近性通过基于特制损失函数的对比学习机制进一步强化。TPDR还利用基于句法特征的第二步重排序,这对某些被Transformer忽略的产品精确匹配(如型号、尺寸)至关重要。为评估方案效果,我们采用来自实际企业的11个数据集,涵盖不同应用场景。实验表明,该方法在71%的案例中能将正确标准化产品检索至前5位,并在80%的案例中将正确类别置于首位。此外,相较于纯句法或纯语义基线方法,其效能提升最高达3.7倍,可解决单独使用任一方法无法处理的案例。