Chinese Legal Case Retrieval (LCR) benchmarks grade a reference judgment relevant when its legal characterization matches the query, and strong systems now reach NDCG@10 of 0.85-0.88. Most of the BM25-to-best-trained gap is recoverable with no retrieval model: ranking candidates only by shared primary charge, broken by BM25, closes 99.2% of it on LeCaRDv2 -- with no detectable difference from the best-trained system. This reflects benchmark design: LeCaRDv2 defines top relevance via the crime's key constitutive elements, which encode the charge, so same-charge cases are relevant by construction (relevance lift 4.49; charge-to-relevance macro-AUC 0.871). Holding charge fixed, the trained reranker's advantage over BM25 collapses to a small within-charge residual (+0.026 NDCG@10, cluster-bootstrap CI excluding zero, about a quarter), the only non-definitional positive. The effect is not uniform: the same rule recovers 84.3% on LeCaRDv1 and is out of spec on CAIL2022, with the charge-to-relevance signal weakening in step (macro-AUC 0.871/0.759/0.728); a predicted-charge cascade reproduces 76.6% on LeCaRDv2 but does not transfer. The construct is also cashable at first stage: an exploratory zero-training charge-pool channel lifts LeCaRDv2 recall (R@100 +0.025, wrong-charge controls hurt), reported as a positive control for the confound, not a retrieval method or novelty claim. Charge is thus a high-leverage construct-validity factor at the benchmark level -- not auniform explanation of NDCG@10, and not evidence that any system relies on charge. We package established construct-validity and partial-input checks as a reusable charge-controlled protocol (CCE); on all three benchmarks its triggers come back null or descriptive, behaving as designed. We release the scripts, schema, and protocol so future benchmarks can be screened before their NDCG@10 is read as legal-reasoning ability.
翻译:中国法律案例检索(LCR)基准通过将法律定性匹配查询的参考判决标记为相关,当前强系统的NDCG@10已达0.85-0.88。无需任何检索模型即可弥合BM25与最优模型之间的大部分差距:仅依据共享主罪名对候选案例排序(并以BM25打破平局),在LeCaRDv2上即可填补99.2%的差距——且与最优系统无显著可检测差异。这反映了基准设计:LeCaRDv2通过犯罪的关键构成要件定义最高相关性,而构成要件编码了罪名信息,因此同罪名案件在定义上即构成相关(相关性提升4.49;罪名-相关性的宏观AUC为0.871)。在固定罪名的情况下,训练后的重排序模型相较于BM25的优势坍缩为微小的同罪名残差(NDCG@10提升+0.026,聚类自助法置信区间排除零值,约占原优势的四分之一),这是唯一非定义性的正向增益。该效应并非均匀分布:相同规则在LeCaRDv1上恢复84.3%,在CAIL2022上则超出规范范围,罪名-相关性信号强度呈阶梯式递减(宏观AUC分别为0.871/0.759/0.728);基于预测罪名的级联方法在LeCaRDv2上复制了76.6%的效果,但无法跨基准迁移。该构念在第一阶段同样可兑现:探索性的零训练罪名池通道将LeCaRDv2的召回率提升(R@100 +0.025,错误罪名控制组出现下降)。此处将其作为混杂变量的阳性对照报告,而非检索方法或新颖性声明。综上,罪名在基准层面构成高杠杆的构念效度因素——但并非NDCG@10的统一解释,也不表明任何系统依赖于罪名。我们将成熟的构念效度检验与部分输入检验整合为可复用的罪名控制协议(CCE);在三个基准上,其触发条件均返回空值或描述性结果,与设计预期一致。我们公开脚本、模式及协议,以便未来基准在将其NDCG@10解读为法律推理能力前接受筛查。