Large language models (LLMs) show growing promise in specialty medicine, yet existing evaluations largely focus on isolated clinical encounters rather than assessing the distinct clinical responsibilities along longitudinal care pathways. We introduce NPC-RubricBench, a phase-aware benchmark for evaluating multistage clinical reasoning across the cancer care continuum, using nasopharyngeal carcinoma as a defined disease context. Curated from real-world records of 405 patients at two cancer centers, it comprises 941 evaluation units across six sequential care stages, from work-up through follow-up, and 17,688 patient-specific criteria across five dimensions. Each assessment is restricted to clinical evidence available at the corresponding decision point. Across 30 general-purpose and medical LLMs, we find strong diagnostic scores did not ensure reliable downstream care. Decision Quality ranged from 56.8 to 94.7 in diagnosis on a 0-100 scale, but dropped to 8.9-56.0 in staging and spanned 32.4-94.2 in treatment. Crucially, failure modes differed across phases: explicit decision errors predominated in anatomical staging (48.2%), whereas decision omissions were more frequent during initial work-up (33.4%) and post-treatment follow-up (30.4%). Varying query granularity revealed a trade-off between decision quality and other clinical requirements. Broad open-ended requests reduced decision quality across all 30 models, whereas more detailed, checklist-like prompts improved decision quality but were accompanied by lower evidence faithfulness and safety adherence. These findings highlight the need to assess clinical reliability against the distinct clinical requirements of each phase along the longitudinal care pathway. Finally, NPC-RubricBench informs the design of stage-aware clinical guardrails and offers a reusable blueprint for evaluating LLMs across other cancers and chronic diseases.
翻译:暂无翻译