Large-scale multi-tenant retrieval systems amass vast user query logs yet critically lack the curated relevance labels required for effective domain adaptation. This "dark data" problem is exacerbated by the operational cost of model updates: jointly fine-tuning query and document encoders requires re-indexing the entire corpus, which is prohibitive in multi-tenant environments with thousands of isolated indices. To address these dual challenges, we introduce \textbf{DevRev Search}, a passage retrieval benchmark for technical customer support constructed through a fully automatic pipeline. We employ a \textbf{fusion-based candidate generation} strategy, pooling results from diverse sparse and dense retrievers, and utilize an LLM-as-a-Judge to perform rigorous \textbf{consistency filtering} and relevance assignment. We further propose a practical \textbf{Index-Preserving Adaptation} strategy: by fine-tuning only the query encoder via Low-Rank Adaptation (LoRA), we achieve competitive performance improvements while keeping the document index frozen. Our experiments on DevRev Search and SciFact demonstrate that targeting specific transformer layers in the query encoder yields optimal quality-efficiency trade-offs, offering a scalable path for personalized enterprise search.

翻译：大规模多租户检索系统积累了海量用户查询日志，却严重缺乏有效领域适配所需的标注相关性标签。这一"暗数据"问题因模型更新的操作成本而加剧：联合微调查询和文档编码器需要重新索引整个语料库，这在拥有数千个独立索引的多租户环境中是难以实现的。为应对这双重挑战，我们引入了\textbf{DevRev Search}——一个通过全自动流水线构建的技术客户支持段落检索基准。我们采用\textbf{基于融合的候选生成}策略，汇集来自多样化稀疏与稠密检索器的结果，并利用LLM-as-a-Judge执行严格的\textbf{一致性过滤}和相关性标注。我们进一步提出实用的\textbf{索引保持适配}策略：通过仅使用低秩适配（LoRA）微调查询编码器，在保持文档索引冻结的同时实现了具有竞争力的性能提升。我们在DevRev Search和SciFact数据集上的实验表明，针对查询编码器中特定Transformer层进行优化可获得最佳的质量-效率权衡，为个性化企业搜索提供了可扩展的路径。