Due to the dynamically evolving nature of real-world query streams, relevance models struggle to generalize to practical search scenarios. A sophisticated solution is self-evolution techniques. However, in large-scale industrial settings with massive query streams, this technique faces two challenges: (1) informative samples are often sparse and difficult to identify, and (2) pseudo-labels generated by the current model could be unreliable. To address these challenges, in this work, we propose a Self-Evolving Relevance Model approach (SERM), which comprises two complementary multi-agent modules: a multi-agent sample miner, designed to detect distributional shifts and identify informative training samples, and a multi-agent relevance annotator, which provides reliable labels through a two-level agreement framework. We evaluate SERM in a large-scale industrial setting, which serves billions of user requests daily. Experimental results demonstrate that SERM can achieve significant performance gains through iterative self-evolution, as validated by extensive offline multilingual evaluations and online testing.

翻译：由于现实世界查询流的动态演化特性，相关性模型难以泛化至实际搜索场景。自演化技术是一种复杂解决方案，但在处理海量查询流的大规模工业环境中，该技术面临两大挑战：(1) 信息丰富的样本通常稀疏且难以识别；(2)当前模型生成的伪标签可能不可靠。为应对这些挑战，本研究提出自演化相关性模型方法（SERM），该方法包含两个互补的多智能体模块：多智能体样本挖掘器（用于检测分布偏移并识别信息丰富的训练样本）和多智能体相关性标注器（通过两级一致性框架提供可靠标签）。我们在日处理数十亿用户请求的大规模工业环境中评估SERM，实验结果表明：经离线多语言评估与在线测试验证，SERM能通过迭代自演化实现显著的性能提升。