Cloud LLM fine-tuning platforms increasingly serve RLHF workloads, where a learned reward model is optimized as a proxy for human quality. As Gao et al. (2023) showed, this proxy diverges from world feedback (downstream eval metrics) under sustained optimization pressure, a phenomenon known as reward overoptimization. Existing platform schedulers ignore this divergence: non-clairvoyant schedulers optimize JCT without any quality signal, SLAQ-style quality-aware schedulers use training loss (a weaker proxy that drops monotonically through hacking), and classical per-job early stopping requires human monitoring and does not free shared GPUs. We propose EvalStop, a composable scheduling primitive that terminates jobs on k consecutive eval-score declines, releases GPUs, preserves the best checkpoint, and delegates to any base scheduler. We frame scheduler-level early stopping as a detection problem and evaluate it in a discrete-event simulator whose RLHF workload mixes reward-hacking and structurally healthy runs, with ground-truth labels hidden from schedulers. On RLHF-heavy workloads (80% RLHF, 64 GPUs), EvalStop achieves precision 98% / recall 99% / FPR 1.5% while improving JCT by 9% and cutting wasted compute by 22% over SRTF-Est (p<0.05). Trivial fixed-progress and loss-plateau competitors either incur 65% FPR on healthy RLHF or miss over half of true hacking cases. Gains compose across every base scheduler tested (9-25% JCT) and detection quality stays stable under eval noise (precision at least 91% at noise std <= 0.05) and hacking base rate (precision at least 89% across 20-80% hacking fractions).

翻译：云端大语言模型微调平台日益承载RLHF（基于人类反馈的强化学习）工作负载，此类任务通过优化学习到的奖励模型来代理人类质量评价。如Gao等人（2023年）所示，在持续优化压力下，该代理模型会偏离世界反馈（下游评估指标），这种现象称为奖励过度优化。现有平台调度器忽视此类偏离：非先知型调度器仅优化作业完成时间而无质量信号；SLAQ类质量感知调度器使用训练损失（一种更弱的代理指标，因作弊行为单调递减）；传统每作业早停机制需人工监控且无法释放共享GPU。我们提出EvalStop——一种可组合的调度原语，在连续k次评估分数下降时终止作业、释放GPU、保留最佳检查点，并委托给任意基础调度器。我们将调度器层面的早停抽象为检测问题，并在离散事件模拟器中评估该机制，其RLHF工作负载混合了奖励作弊与结构健康运行，且真实标签对调度器不可见。在RLHF密集型工作负载（80% RLHF，64 GPU）下，EvalStop相较SRTF-Est实现了98%精准率/99%召回率/1.5%假阳性率，同时将作业完成时间提升9%，减少22%算力浪费（p<0.05）。简单固定进度法与损失平台竞争方案或对健康RLHF产生65%假阳性率，或漏检半数以上真实作弊案例。该增益在所有测试的基础调度器上均具有可组合性（作业完成时间提升9-25%），且检测质量在评估噪声（噪声标准差≤0.05时，精准率≥91%）与作弊基础发生率（20-80%作弊比例区间内，精准率≥89%）条件下保持稳定。